Steve_in_Texas wrote on Nov 8^th, 2010 at 4:51pm:


Sesame is a 32 bit application. As such, it can only use 2 GB on Windows and 4 GB on Linux. If you have 8 GB, it becomes more likely that it can use that full amount, because it need not share as much with other applications, the OS, etc. Are you importing/exporting 10,000 total records, or 10,000 parent records (and many children per parent)?

Quote:


Sesame server will use one "main" thread, two additional threads per connected client, and then a new thread for each command issued to the server. It is up to the OS to allocate processors to these threads. It may be that the communications threads are running on separate processors, but are waiting around for messages to come in, so they don't use much CPU time. Identical commands, like those that comprise an import or an export, can't run in parallel. So you may be waiting, if you have lengthy commands that everyone is attempting to run at the same time. The more diverse the activity, the more parallel Sesame server will become, because you don't have different command threads competing for the same resources, and having to wait for each other.

Steve_in_Texas wrote on Nov 9^th, 2010 at 11:30pm:


How much actually data is exported, in bytes? How many fields per record?

Quote:


There is a 64 bit version of Sesame 3.0. No release date yet. 64 bits will allow you to make better use of your RAM, which in your case will speed things up. Are you swapping?

Steve_in_Texas wrote on Nov 10^th, 2010 at 3:42pm:


A flat database shouldn't be accumulating much memory at all on an export. There must be something else going on here. Was this database ever not flat?

You could watch the server process in task manager or one of the sysinternals tools while it is performing the export to see how its memory usage is expanding.

If you believe that there is a structural problem, you could dump all of the data to a text file, retranslate/rebuild the application with no data, then reload the data. Make a backup first.

If you wish, I could take a look at it this weekend. Compress the .db and .dat files using zip and email them to me (laser@HammerVE.com) or support (support@lantica.com).

I ran some preliminary tests and don't see any significant or out-of-scale memory usage. Did you try monitoring it on your server using task manager? Keep your eyes open for any task (not just Sesame) that is swelling its memory usage as Sesame is exporting. Are you running any kind of file scanner (like a virus scanner) that might be reacting poorly while a file is being written? Also, during the export, there are two stages. One while it prepares for export and then the actual export. During which of these stages does the memory event happen?

Mark, Task manager had nothing unusual to report. During the export of all fields, for 1/2 of the records in my database, Sesame memory usage climbed from 430,000K to 550,000K, then dropped back down when finished.

The performance meters showed total CPU usage move from 4% to 20%, then back to 4%. PF Usage seems to remain steady at 1.99 GB.

There is not much else running on this server, and no programs 'swelled' in memory usage during the import/export. Also, I did not get the 'memory low' message today. Perhaps the weekend reboot kept the issue at bay.

I'll monitor it throughout the week.

Steve

update again:

on the Server computer, in standalone mode: I exported all records (2 batches to avoid a crash.) Deleted all records from sesame. Imported all records back in (both batches). Tried to export ALL records...crashed!

(Server log shows 'out of memory')

I give up.