Normal Topic Doing a backup on an actively served app. (Read 3044 times)
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Doing a backup on an actively served app.
Apr 22nd, 2006 at 12:38pm
Print Post Print Post  
Most of this discussion only applies to Sesame 1.1.x - Sesame 2.0 knows how to do uniquely named backups based on different conditions.

There has been some discussion about how to do a backup of Sesame 1.1.x application files that are in active use when the backup has to happen. This was asked by a Linux user a while ago and I worked out the general logic for a shell script (the Unix equiv. of a "batch file") that could take the necessary steps.

I have removed from the example below the Linux only terminology and stated everything as just logical steps:

Code
Select All
Get the process ID for Sesame
while not done and have not tried enough
  check if Sesame has application files open
  if the files are not open
    suspend the Sesame process
    copy the .db and .dat to new and unique files
    unsuspend the Sesame process
    set done to true
  else
    wait a short while (less than 10 seconds)
  end if
end while
 



A script, like the one above could be launched at intervals to create backups throughout the day, week, month, etc...

In Linux, there are command line programs that can handle each of these steps:

"pidof" - returns the process ID of a process
"fuser" and "lsof" - can tell if a file is open by a process
"kill" - can suspend and resume a process
"cp" - can copy a file to a unique filename (with a little help from the shell script to make the filename

I am sure that there are command line utilities in Windows that can perform each of the steps, but I don't know what they are. If anyone knows of any, please post them here.

If we can't find the same sort of utilities for Windows, I'll probably write a small utility for Windows that can ask Sesame Server for the info and do a file copy on interval for 1.1.x. But, I'd rather not duplicate functionality that is either in the OS (or should be), or readily available from reputable sources.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: Doing a backup on an actively served app.
Reply #1 - Apr 22nd, 2006 at 9:12pm
Print Post Print Post  
XP and newer versions of Windows have TASKLIST and TASKKILL commands.

Try PrcView at http://www.teamcti.com/pview/prcview.htm
This is a FREE utility.   Works on ALL versions of Windows.
No complex install, just copy about 5 small files to a folder.

For many years, I have used the GUI version running in systray from StartUp.   Can get to things that CTL-ALT-DEL does not show. 

But command line tools also exist that can be used in batch files..

From their documentation: Quote:
Introduction

PrcView is a process viewer utility that displays detailed information about processes running under Windows. For each process it displays memory, threads and module usage. For each DLL  it shows full path and version information. PrcView comes with a command line version that allows you to write scripts to check if a process is running, kill it, etc.

What’s new in 5.0

    * Improved GUI and usability
    * Functional enhancement in many areas
    * Show CPU and memory consumption for the selected processes
    * Show system entries that are automatically executed at the start-up
    * Includes process monitoring functionality

There is no auto notification of upgrades, so you need to check every few months for updates.

The current version 5.2.15.1 has excellent tools for monitoring processes real time, and a great StartUp hierarchical tree. 

Some of the command line switches also allow scheduling and repeating and delays.  Too many switches to cover here.

« Last Edit: Apr 22nd, 2006 at 11:17pm by Bob_Hansen »  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Doing a backup on an actively served app.
Reply #2 - Apr 23rd, 2006 at 1:11am
Print Post Print Post  
The folks at www.sysinternals.com (the same folks that exposed the Sony "roothack" on Sony produced music CDs) have several utilities that look like they might be useful. Among them the "ps" utilities, especially psSuspend, that can suspend and resume a process by process name, looks especially likely.

Has anyone looked at any of these?
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: Doing a backup on an actively served app.
Reply #3 - Apr 23rd, 2006 at 4:21am
Print Post Print Post  
Hi Mark ...

I have used some of the SysInternals utilities over the past three years.  But one thing to point out is that many of the commands only work on Windows XP/NT or newer, not on WIN9x systems.   Since I support clients with many older systems, I prefer to use tools that I can use on all systems.

SysInternals has many more tools than PrcView, because PrcView is specialized for that single function, contolling processes.  SysInternals has more tools because it is a toolbox utility package.

The PsTools section does contain utilities to ID procceses and to kill them, but this is one of the utility sets that does NOT run on older systems.  So, for this function,  I prefer PrcViewer.  I also like the GUI capability in the systray in addition to the command line.  I use that constantly vs. CTL-ALT-DEL, three fingered salute.

And I think that you will also find that the PrcView command line switches have more capability than PsTools, but it has been a while since I looked at them.  If I get a chance, I will try to compare them both.


  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Doing a backup on an actively served app.
Reply #4 - Apr 23rd, 2006 at 1:24pm
Print Post Print Post  
Ah. I'm not arguing the case of any tool over another. But with very specific needs within this script, I figure the more utilties we look at, the better. For example, the utility that returns the process ID, needs to be able to return that into a script variable, or the utility that suspends the Sesame process must be able to do so by name (which may not be specific enough, given that the server and the client may be running on the same box and both use the same name). With needs that specific, it is probably best to find the widest range of command line tools as we can.

My temptation is to simply go with one of the Unix command sets that are available to Windows. Cygwin or AT&T, etc... provide fairly complete Unix command sets for MS OSs. But these are a bit overkill (600-1200 commands each) when we need only four of the commands.

When I get to the office, where I have a MS box, I will certainly take a look at PrcView, as well as any other utilities that may get mentioned. It appears that there may be some similar tools available from MS themselves.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: Doing a backup on an actively served app.
Reply #5 - Apr 23rd, 2006 at 5:24pm
Print Post Print Post  
I could not locate list/kill in my copy of Cgywin, too many functions there. 

But here is the syntax for Microsoft / PrcView / PsTools List/Kill functions.  Again note that only PrcView works on all Windows, others need XP or higher.

=======================
TaskList: Quote:
TASKLIST [/S system [/U username [/P [password]]]]
        [/M [module] | /SVC | /V] [/FI filter] [/FO format] [/NH]

Description:
    This command line tool displays a list of application(s) and
    associated task(s)/process(es) currently running on either a local or
    remote system.

Parameter List:
   /S     system           Specifies the remote system to connect to.

   /U     [domain\]user    Specifies the user context under which
                          the command should execute.

   /P     [password]       Specifies the password for the given
                          user context. Prompts for input if omitted.

   /M     [module]         Lists all tasks that have DLL modules loaded
                          in them that match the given pattern name.
                          If the module name is not specified,
                          displays all modules loaded by each task.

   /SVC                    Displays services in each process.

   /V                      Specifies that the verbose information
                          is to be displayed.

   /FI    filter           Displays a set of tasks that match a
                          given criteria specified by the filter.

   /FO    format           Specifies the output format.
                          Valid values: "TABLE", "LIST", "CSV".

   /NH                     Specifies that the "Column Header" should
                          not be displayed in the output.
                          Valid only for "TABLE" and "CSV" formats.

   /?                      Displays this help/usage.

Filters:
    Filter Name     Valid Operators           Valid Value(s)
    -----------     ---------------           --------------
    STATUS          eq, ne                    RUNNING | NOT RESPONDING
    IMAGENAME       eq, ne                    Image name
    PID             eq, ne, gt, lt, ge, le    PID value
    SESSION         eq, ne, gt, lt, ge, le    Session number
    SESSIONNAME     eq, ne                    Session name
    CPUTIME         eq, ne, gt, lt, ge, le    CPU time in the format
                                             of hh:mm:ss.
                                             hh - hours,
                                             mm - minutes, ss - seconds
    MEMUSAGE        eq, ne, gt, lt, ge, le    Memory usage in KB
    USERNAME        eq, ne                    User name in [domain\]user
                                             format
    SERVICES        eq, ne                    Service name
    WINDOWTITLE     eq, ne                    Window title
    MODULES         eq, ne                    DLL name

Examples:
    TASKLIST
    TASKLIST /M
    TASKLIST /V
    TASKLIST /SVC
    TASKLIST /M wbem*
    TASKLIST /S system /FO LIST
    TASKLIST /S system /U domain\username /FO CSV /NH
    TASKLIST /S system /U username /P password /FO TABLE /NH
    TASKLIST /FI "USERNAME ne NT AUTHORITY\SYSTEM" /FI "STATUS eq running"


========================
TaskKill: Quote:
TASKKILL [/S system [/U username [/P [password]]]]
        { [/FI filter] [/PID processid | /IM imagename] } [/F] [/T]

Description:
    This command line tool can be used to end one or more processes.
    Processes can be killed by the process id or image name.

Parameter List:
    /S    system           Specifies the remote system to connect to.

    /U    [domain\]user    Specifies the user context under which
                          the command should execute.

    /P    [password]       Specifies the password for the given
                          user context. Prompts for input if omitted.

    /F                     Specifies to forcefully terminate
                          process(es).

    /FI   filter           Displays a set of tasks that match a
                          given criteria specified by the filter.

    /PID  process id       Specifies the PID of the process that
                          has to be terminated.

    /IM   image name       Specifies the image name of the process
                          that has to be terminated. Wildcard '*'
                          can be used to specify all image names.

    /T                     Tree kill: terminates the specified process
                          and any child processes which were started by it.

    /?                     Displays this help/usage.

Filters:
    Filter Name   Valid Operators           Valid Value(s)
    -----------   ---------------           --------------
    STATUS        eq, ne                    RUNNING | NOT RESPONDING
    IMAGENAME     eq, ne                    Image name
    PID           eq, ne, gt, lt, ge, le    PID value
    SESSION       eq, ne, gt, lt, ge, le    Session number.
    CPUTIME       eq, ne, gt, lt, ge, le    CPU time in the format
                                           of hh:mm:ss.
                                           hh - hours,
                                           mm - minutes, ss - seconds
    MEMUSAGE      eq, ne, gt, lt, ge, le    Memory usage in KB
    USERNAME      eq, ne                    User name in [domain\]user
                                           format
    MODULES       eq, ne                    DLL name
    SERVICES      eq, ne                    Service name
    WINDOWTITLE   eq, ne                    Window title

NOTE: Wildcard '*' for the /IM switch is accepted only with filters.

NOTE: Termination of remote processes will always be done forcefully
     irrespective of whether /F option is specified or not.

Examples:
    TASKKILL /S system /F /IM notepad.exe /T
    TASKKILL /PID 1230 /PID 1241 /PID 1253 /T
    TASKKILL /F /IM notepad.exe /IM mspaint.exe
    TASKKILL /F /FI "PID ge 1000" /FI "WINDOWTITLE ne untitle*"
    TASKKILL /F /FI "USERNAME eq NT AUTHORITY\SYSTEM" /IM notepad.exe
    TASKKILL /S system /U domain\username /FI "USERNAME ne NT*" /IM *
    TASKKILL /S system /U username /P password /FI "IMAGENAME eq note*"


=============================
=============================
PrcView: Quote:
Usage: pv -[<MODE>] -[<OPTIONS>] <ARGUMENTS>...-[<OPTIONS>]

Modes:
  -s       --summary   show usage for the specified MODULE
  -h,-?    --help      display this help information

Actions:
  -k       --kill      kill process
  -a       --activate  brings process main window in the foreground
  -c       --close     close (send WM_CLOSE) to the PROCESS
  -p[nihr] --priority  set priority to "Normal", "Idle", "High", "Real Time"
    [ba]               "Below Normal" and "Above Normal" only on W2K or higher

Output Options:
  -e,      --extend    show additional information if available
  -l,      --long      show command line (can also be a filter)
  -q,      --quiet     supress headers and produce a tab-separated list
  -b       --bare      show process ID only ()

Input Options:
  -f,      --force     never prompt
  -i,      --id        use process ID instead of the PROCESS name

Filters:
  -l[mask] --long      include processes with command line matching mask
  -w[mask] --window    show processes with visible windows matching mask,
                      -e includes in search also invisible windows
  -u[mask] --usage     show processes using modules that matches mask
  -t[root] --tree      display process tree starting starting from the root

Extra Information Options:
  -g       --getenv    get startup environment for the  PROCESS
  -m       --module    show modules used by specified PROCESS

Execution Options:
  -d[time] --delay     delay time in milliseconds before executing command
  -r[err]  --repeat    repeat command in a cycle, while (%ERRORLEVEL% > err)
  -n       --number    %ERRORLEVEL% = negated number of matched processes
  -x[a]    --exit      wait for the process completion (exit)
                       'a' flag waits for all processes, -d sets time-out
  -@[file_name]        read arguments from specified file or from
                      standard input after processing the command line

Arguments can contain '*' and '?' wildcards.

Use return code (%ERRORLEVEL%) in batch files:
    0 - process found (negated number of processes if -n is specified)
    1 - empty result set, 2 - programm error

Examples:
  pv myprocess.exe        get process ID for myprocess.exe.
  pv -e                   get extended list of running processes.
  pv -k sleep*            kill all processes starting with "sleep"
  pv -m -e explorer.exe   get extended information about explorer's modules
  pv -u oleaut*.dll       list of all processes that use matching dll
  pv -ph w*.exe           set priority to hight for all matching processes
  pv explorer.exe -l"*/S" looks for explorer process with /S switch
  pv -r0 -d2000 calc.exe "2>nul" checks every 2 seconds if calc.exe is running


============================
============================
PSLIST: Quote:
usage: pslist [-?] [-d] [-m] [-x][-t][-s [n] [-r n]  [\\computer [-u username] [-p password]] [name | pid]

-d This switch has PsList show statistics for all active threads on the system, grouping threads with their
owning process.

-m This switch has PsList show memory-oriented information for each process, rather than the default of CPU-
oriented information.

-x

With this switch PsList shows CPU, memory and thread information for each of the processes specified.

-t Show process tree.

-s [n] Run in task-manager mode, for optional seconds specified.

Press Escape to abort.

-r n Task-manager mode refresh rate in seconds (default is 1).

-u Specifies optional user name for login to remote computer.

-p Specifies optional password for user name. If you omit this you will be prompted to enter a hidden
password.

name Instead of listing all the running processes in the system, this parameter narrows PsList's scan to
those processes that begin with the name process. Thus:

pslist exp

would statistics for all the processes that start with "exp", which would include Explorer.

pid Instead of listing all the running processes in the system, this parameter narrows PsList's scan to the
process that has the specified PID. Thus:

pslist 53

would dump statistics for the process with the PID 53.

===========================
PSKILL: Quote:
usage: pskill [\\computer [-u username] [-p password]] <process name | process id>

-u
Specifies optional user name for login to remote computer.

-p
Specifies optional password for user name. If you omit this you will be prompted to enter a hidden password.

process id
Specifies the process ID of the process you want to kill.

process name
Specifies the process name of the process or processes you want to kill.


=============================


  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Doing a backup on an actively served app.
Reply #6 - May 12th, 2006 at 4:06pm
Print Post Print Post  
Thanks for all the info regarding the new backup options in Sesame 2.0. Sounds very exciting. Regarding a current solution for 1.1.3, I'm still a bit confused.

What would happen if we use the 'backup application' from within sesame while another user has a form open. Would trouble occur ONLY if the form was currently being written to during the backup procedure? If everyone takes their hands off the keyboard, can a reliable backup be made at that time?

If using a third party program, like Second Copy to make an hourly back up of the .DSR and .DB files while the application is open and in use, will we cause damage to the open application? Will ALL of the backups made using this techniqe be corrupt and locked?

Currently, I have Second Copy making a backup nightly on an application that is being served, but all users are gone, which seems to be working fine, but I would prefer an hourly backup routine.

Thanks,
Steve
  
Back to top
IP Logged
 
Ray the Reaper
Global Moderator
Members
Lantica Support
*****
Offline


The One & The Only

Posts: 2480
Joined: Aug 20th, 2003
Re: Doing a backup on an actively served app.
Reply #7 - May 12th, 2006 at 5:07pm
Print Post Print Post  
Quote:
Thanks for all the info regarding the new backup options in Sesame 2.0. Sounds very exciting. Regarding a current solution for 1.1.3, I'm still a bit confused.

What would happen if we use the 'backup application' from within sesame while another user has a form open. Would trouble occur ONLY if the form was currently being written to during the backup procedure? If everyone takes their hands off the keyboard, can a reliable backup be made at that time?



Using the "Backup Application" choice from the menu tree will create an exact copy of the application as it is loaded into Memory on the Server. It will not be locked. As far as I know a reliable Backup can be made this way even if people are typing away on records. Some of the Sesame Clients will pause on certain operations while the backup is being made. So while a backup is being made the user can type data into an element but when they hit F10 it will appear to be waiting for something. Once the Backup is complete, the actions will resume.

Since Backup Application saves the Application as it is loaded into Memory you could create a good copy of your Sesame file by using this method. This would be useful if the files on the hard drive have been corrupted (Someone exported over the DAT file, Reconciled the DB while it was loaded, Hard Drive where the data is being stored is acting funky, etc.)

Quote:
If using a third party program, like Second Copy to make an hourly back up of the .DSR and .DB files while the application is open and in use, will we cause damage to the open application? Will ALL of the backups made using this techniqe be corrupt and locked?


It is unlikely that it will corrupt the open Application, but it could. All the backups made using this application will be locked and they might be corrupted. It all depends on timing. If no one is saving any records(they back two feet away from their computer), while Second Copy is making the backup then both the original file and the copy will be fine. The copy will however be locked as the original file is also locked. If however records are being changed and committed then it all depends on the timing of the copy. For example Second Copy could copy the DB file BEFORE the record was changed/added and it could copy the DAT file AFTER the record was changed/added. At this point the copied DB and DAT files would not match. This is why, to be absolutely positive that the backup works, you need to suspend the Sesame Server, copy the files, then unsuspend the Sesame Server.

In my opinion, The risk to both the open Application and the backup files is too great to attempt a backup(by just copying the files) while the Sesame Server is running and could, at any time, write to the open files.

Quote:
Currently, I have Second Copy making a backup nightly on an application that is being served, but all users are gone, which seems to be working fine, but I would prefer an hourly backup routine.

Thanks,
Steve


For an hourly backup routine you may want to consider a batch file in scheduled tasks that opens a Sesame Client with a macro. The macro loads the application and backs it up to a set name. Then the macro closes the Sesame Client. The batch file then needs to move that backup(It will always be in the same spot with the same name) to a new directory or possibly just a new name.

To do an External Hourly backup the backup process has to follow the steps as Mark outlined them above. Otherwise the backup may not be a good backup.

-Ray
  

Raymond Yoxall Consulting
ray.yoxall@gmail.com
ryoxall@lantica.com
Sesame Applications, Design and Support
Back to top
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Doing a backup on an actively served app.
Reply #8 - May 12th, 2006 at 5:36pm
Print Post Print Post  
Ray,

EXCELLENT info. You answered all my questions and made it easy to understand and even taught me a few extra things about how Sesame works.

We still struggle with getting the Sesame server to properly shut down and restart (we often get "could not create socket" becuase it thinks users are still connected) so it sounds like your macro suggestion could be ideal for an hourly backup.

We'll try that until ver 2.0 is available.

Thanks again,
Steve
  
Back to top
IP Logged