Normal Topic Client out of memory??? (Read 1464 times)
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Client out of memory???
Aug 12th, 2004 at 12:20am
Print Post Print Post  
During a programmed export of 1300 record with about 3000 (total) subrecords, the program aborted due to "Client out of memory" about 90% of the way through.

The program is supposed to write the field data to a csv file as it reads through the records. Does this error mean that there is some data that is being 'building up' in memory as it reads through the records?

I thought that data could not be retained between records unless it is written as a static program or in the global values section.

Would adding 'variantname=""' for all variants at the end of the program help to free up memory and avoid this problem?

Thanks

Steve
  
Back to top
IP Logged
 
Hammer
YaBB Administrator
Lanticans
*****
Offline


Fire bad. Tree pretty.

Posts: 3436
Location: Ohio
Joined: Nov 22nd, 2002
Re: Client out of memory???
Reply #1 - Aug 12th, 2004 at 11:00am
Print Post Print Post  
Steve,

Are you doing an Export, or are you running SBasic in a Mass Update that  uses the File I/O commands?

If you are using SBasic, can I see the SBasic?
  

- Hammer
The plural of anecdote is not data.
Back to top
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Client out of memory???
Reply #2 - Aug 12th, 2004 at 1:21pm
Print Post Print Post  
I'm using SBasic and the progrom is VERY large and quite sloppy becuase i haven't learned how to use subroutines. Its a plain, top to bottom program. I'll email it to you. (There may be about 60 variables assigned)

I looked it over carefully, and dont see where a variable would retain information between records. (I am using no statics or global values) Smaller exports produce perfect .csv's (Thanks Sesame!)

Most variables write to the .csv file and others are used for calculations, etc and all my data in the csv files are correct, so they have to be clearing out after each record (like their supposed to). My guess is I am not using filehandles properly and they are holding in memory as well as writing to disk.

The .csv file is getting to about 7 MB before the client runs out of memory (The process takes about an hour on our 10/100 LAN) the client has 256 MB of memory

I got to 10MB using our server as the client with has 1GB of memory. It crashed but I think becuase another user was working in the same db during the export.

Thanks,
Steve
  
Back to top
IP Logged
 
Hammer
YaBB Administrator
Lanticans
*****
Offline


Fire bad. Tree pretty.

Posts: 3436
Location: Ohio
Joined: Nov 22nd, 2002
Re: Client out of memory???
Reply #3 - Aug 12th, 2004 at 2:27pm
Print Post Print Post  
Steve,

I got your code. It's big enough that it's going to take me little while to sort through it.

BTW, I would suggest that you put this code in a Mass Update, rather than using FormResultSetCurrentPosition to move between the parent records yourself. You can use @SelectTreeItem and @LoadMassUpdateSpec to run the saved Mass Update. You can also use Global Code and @ResultSetCurrentPosition to do things only on the first and/or last record of the Mass Update. That way, Sesame can handle advancing the parent records for you.
  

- Hammer
The plural of anecdote is not data.
Back to top
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Client out of memory???
Reply #4 - Aug 12th, 2004 at 2:41pm
Print Post Print Post  
Erika,

No hurry on looking at my code. I'm doing my export in small batches and its working beautifully. I am always so impressed with sesame once the .csv if complete. Smiley


The code exports only the retrieved batch, and the advancing of records is done with programming.  The user does not have to hit the command button on every record.

Should I still consider mass update?

Steve
  
Back to top
IP Logged
 
Hammer
YaBB Administrator
Lanticans
*****
Offline


Fire bad. Tree pretty.

Posts: 3436
Location: Ohio
Joined: Nov 22nd, 2002
Re: Client out of memory???
Reply #5 - Aug 12th, 2004 at 2:57pm
Print Post Print Post  
Yes. Operating only on a retrieved batch of records is what Mass Update lives for. User clicks the command button on any record, and the Mass Update exports the whole batch and returns you to the starting point.
  

- Hammer
The plural of anecdote is not data.
Back to top
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Client out of memory???
Reply #6 - Aug 12th, 2004 at 3:07pm
Print Post Print Post  
OK. Sounds like an easier program to write, too.

Thanks for the tip. Always appreciated (and needed).

Again, no hurry on looking at my code, but I'm dying to hear your comments on my export program (it's my 'baby')

Please be gentle with your response. Smiley

Thanks,
Steve
  
Back to top
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Client out of memory???
Reply #7 - Aug 12th, 2004 at 8:28pm
Print Post Print Post  
Erika,

I noticed today that some variants (vtype, for instance) is definately retaining its last value between records. I thought this should not happen when a program such as the one I sent you.

I assume adding 'vtype = ""' at the end, or declaring it such as var vtype as string = "" at the beginning should solve this temporarily, but is this a bug?

Sounds like I need to check all my csv files for errors  Shocked

Thanks,
Steve
  
Back to top
IP Logged
 
Hammer
YaBB Administrator
Lanticans
*****
Offline


Fire bad. Tree pretty.

Posts: 3436
Location: Ohio
Joined: Nov 22nd, 2002
Re: Client out of memory???
Reply #8 - Aug 12th, 2004 at 8:48pm
Print Post Print Post  
Quote:
Erika,

I noticed today that some variants (vtype, for instance) is definately retaining its last value between records. I thought this should not happen when a program such as the one I sent you.

I assume adding 'vtype = ""' at the end, or declaring it such as var vtype as string = "" at the beginning should solve this temporarily, but is this a bug?


No, it's not a bug. You are managing your own record navigation, so you have to clear your own variables. You should always initialize variables, even in a Mass Update, which does know how to reset things based on moving between records. Changing the result set position in code does not have the same auto-effects as normal navigation, nor should it. Otherwise, you wouldn't be able to total up all your subrecords, etc.
  

- Hammer
The plural of anecdote is not data.
Back to top
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Client out of memory???
Reply #9 - Aug 12th, 2004 at 8:56pm
Print Post Print Post  
Thanks, that would solve my issues. (I had a blank record at the end so it got duplicated in the csv.)

I guess you dont need to read through my programming after all. Sorry for the trouble.

Last question. Would Initializing variables at the beginning also clear them out during result set change or do I need vtype = "" at the end ALSO?

Thanks,
Steve


  
Back to top
IP Logged