Page Index Toggle Pages: 1 [2] 3  Send Topic Send Topic Print Print
Very Hot Topic (More than 25 Replies) Combining Elements to Make Unique Ext. Key (Read 4305 times)
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #15 - Apr 7th, 2006 at 7:43pm
Print Post Print Post  
Sesame 2.0 does have this capability in a set of commands called the "XResultSet" commands. Primary is the @XResultSetSearch command:

@XResultSetSearch(filename as string, database_name as string, search_mode as int, search_syntax as int, search_specs as string, ...) as int

For example to search for records in Customers.db, where the Company field starts with "M" and the Key field is less than 1500, you would use:
Code
Select All
var aa as int

aa = @XResultSetSearch("Customers.db", "Customers", 0, 2, "!Key=<1500.0", "!Company=m..")
 



You can have as many search specs as you want.

The search modes are:
  AND = 0
  OR = 1
The search syntax flags:
  Q&A = 2
  REGEX = 3

The integer returned is a "handle" or "key" value that you can use to access the resulting result set using the other XResultSet commands:

               XResultSetClose(rs_key as int)
                       Closes an open external result set.

               XResultSetCurrentPosition(rs_key as int, record_number as int)
               @XResultSetCurrentPosition(rs_key as int) as int
                       Sets/Gets the record number to work on in the external result set.

               @XResultSetTotal(rs_key as int) as int
                       Returns the number of records in the external result set.

               @XResultSetValue(rs_key as int, field_name as string) as string
                       Returns the value of the field specified in "field_name" from the current record in the external result set.

               XResultSetValue(rs_key as int, field_name as string, value as string)
                       Sets the value of the field specified in "field_name" in the current record in the external result set to the
                               value in the parameter "value".

               XResultSetCreateNewRecord(rs_key as int)
                       Creates a new record in the external result set and makes it the current record.

               XResultSetRemoveRecord(rs_key as int)
                       Removes the current record from the external result set - but does not delete the record.

               XResultSetDeleteRecord(rs_key as int)
                       Deletes the current record.

               XResultSetReparent(rs_parent_key as int, field_name as string, rs_child_key)
                       Reparents the records in the child result set under the specified field in the current record of the parent
                               result set.

  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Carl Underwood
Senior Member
Members
*****
Offline



Posts: 1350
Location: New Hampshire
Joined: Mar 11th, 2003
Re: Combining Elements to Make Unique Ext. Key
Reply #16 - Apr 7th, 2006 at 7:59pm
Print Post Print Post  
Quote:
It would be nice if we do not have to go through loops and arrarys.

Well Bharat, it looks like Mark has taken care of your request. It looks like the new commands are just what we need.

Thanks Mark, those look like a great solution! Grin

PS: I guess I won't be needing to write that article after all. Grin
  


Carl Underwood
CDU Computer Consulting LLC
Epsom, New Hampshire
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #17 - Apr 7th, 2006 at 8:06pm
Print Post Print Post  
Quote:
PS: I guess I won't be needing to write that article after all. Grin


Write the article anyways. Its always better to have lots of good solutions, especially if you have to wait for the one above.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: Combining Elements to Make Unique Ext. Key
Reply #18 - Apr 7th, 2006 at 8:21pm
Print Post Print Post  
I can hardly wait for version 2.0. It is going to be one major update. Is Lantica is going to go all out with marketing after version 2.0?  I noticed that Sesame is practically sold through only the website.  It would be nice, if it is distributed through other chanels like computer stores, Amazon.com etc.  Not that it matters to us as users but if Lantica gets more revenue, a part of it will go towards the product enhancement and that will translate into more power to Sesame.

I remember the day when Symantec announced that Q&A was not going to be supported and they were not coming out with anymore new version. It was a major trauma to my psyche. There is always some sort of fear lingering in the background.

Well, I am extremely delighted seeing Sesame making steady progress in terms of adding new features. It is going to be one of a kind software and I am glad we are having early headstart with the significant development.

Bharat Naik

  
Back to top
 
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #19 - Apr 7th, 2006 at 11:44pm
Print Post Print Post  
Hmmm, I caught that unobtrusive reference to RegEx in there Mark.

Can you provide any advance info re syntax, greedy, Posix, etc.  There are many variations avaialble, would just like to be ahead on this one.  Will this be homegrown or based on existing RegEx used in another product?

I have been waiting for this tool, but can already see the need for a separate Forum Topic just to to handle RegEx questions.   

  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #20 - Apr 8th, 2006 at 12:09am
Print Post Print Post  
It is based on the same regex library that has been embedded in Sesame all along (Sesame translates Q&A search syntax into regex before use and always has). This library is AT&T V8 compatible (though not source derived from AT&T), is standard on most releases of Linux and BSD. It was originally written in 1986, but was rewritten for use in C++ (as opposed to C) in 1998. The original author of the base C library was Henry Spencer of the University of Toronto, but it has passed through many hands since. It is POSIX.2 compliant and compatible with egrep syntax.

From the egrep manual page:
Quote:
      The fundamental building blocks are the regular expressions that match a single character.  Most characters, including all letters and digits, are regular expressions that match themselves.  Any metacharacter  with  special  meaning may be quoted by preceding it with a backslash.

      A bracket expression is a list of characters enclosed by [ and ].  It matches any single character in that list; if the first character of the list is the caret ^ then it matches any character not in the list.  For example, the regular expression [0123456789] matches any single digit.

      Within a bracket expression, a range expression consists of two characters separated by a hyphen.  It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set.  For example, in the default C locale, [a-d] is equivalent to [abcd].  Many locales sort characters in dictionary order, and in these locales [a-d] is typically not equivalent to [abcd]; it might be equivalent to [aBbCcDd], for example.  To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the LC_ALL environment variable to the value C.

      Finally, certain named classes of characters are predefined within bracket expressions, as follows.  Their names are self explanatory, and they  are  [:alnum:],  [:alpha:],  [:cntrl:],  [:digit:],  [:graph:],  [:lower:],  [:print:], [:punct:], [:space:], [:upper:], and [:xdigit:].  For example, [[:alnum:]] means [0-9A-Za-z], except the latter form depends upon the C locale and the ASCII character encoding, whereas the former is independent of locale and character set.  (Note that the brackets in these class names are part of the symbolic names, and must be included in addition to the brackets delimiting the bracket list.)  Most metacharacters lose their special meaning inside lists.   To include a literal ] place it first in the list.  Similarly, to include a literal ^ place it anywhere but first.  Finally, to include a literal - place it last.

      The period .  matches any single character.  The symbol \w is a synonym for [[:alnum:]] and \W is a synonym for [^[:alnum]].

      The  caret  ^  and the dollar sign $ are metacharacters that respectively match the empty string at the beginning and end of a line.  The symbols \< and \> respectively match the empty string at the beginning and end of a word.  The symbol \b matches the empty string at the edge of a word, and \B matches the empty string provided it's not at the edge of a word.

      A regular expression may be followed by one of several repetition operators:
      ?      The preceding item is optional and matched at most once.
      *      The preceding item will be matched zero or more times.
      +      The preceding item will be matched one or more times.
      {n}    The preceding item is matched exactly n times.
      {n,}   The preceding item is matched n or more times.
      {n,m}  The preceding item is matched at least n times, but not more than m times.

      Two regular expressions may be concatenated; the resulting regular expression matches any string formed by concatenating two substrings that respectively match the concatenated subexpressions.

      Two regular expressions may be joined by the infix operator |; the resulting regular expression matches any string matching either subexpression.

      Repetition takes precedence over concatenation, which in turn takes precedence over alternation.  A whole subexpression may be enclosed in parentheses to override these precedence rules.

      The backreference \n, where n is a single digit, matches the substring previously matched by the nth parenthesized subexpression of the regular expression.

      In basic regular expressions the metacharacters ?, +, {, |, (, and ) lose their special meaning; instead use the backslashed versions \?, \+, \{, \|, \(, and \).

      Traditional egrep did not support the { metacharacter, and some egrep implementations support \{ instead, so portable scripts should avoid { in egrep patterns and should use [{] to match a literal {.

      GNU egrep attempts to support traditional usage by assuming that { is not special if it would be the start of an invalid interval specification.  For example, the shell command egrep '{1' searches for  the  two-character  string  {1 instead of reporting a syntax error in the regular expression.  POSIX.2 allows this behavior as an extension, but portable scripts should avoid it.


Also, of note: the form based search in Sesame 2.0 optionally supports regex syntax. You can alternate between Q&A and regex syntax, but you cannot use both at the same time on the same form.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #21 - Apr 8th, 2006 at 3:56am
Print Post Print Post  
Thanks Mark.

You have provided some guidelines, but I was wondering about more specifics re Sesame's usage.  For example you provide Quote:
[a-d] is equivalent to [abcd].  Many locales sort characters in dictionary order, and in these locales [a-d] is typically not equivalent to [abcd]; it might be equivalent to [aBbCcDd], for example.
Will Sesame use [a-d] or require [AaBbCcDd]? (Hmm, just noticed the typo in the section from the manual).    Any phrases like [:digit:] to represent [0123456789] or [0-9] , or [:space:] to represent blank spaces? 

I guess I am asking to see an advance of the Sesame manual's section on help for RegEx syntax.  Since you have noted that this will be egrep compatible, then I will use that as a reference.  This could also mean that I can probably use most of any RegEx libraries that I have accumulated.

In addition to using RegEx in XL statements, will there be a RegEx based Search/Replace function for strings?  Or will the existing @Replace just be enhanced to use RegEx?

Thanks again for the preview.
  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #22 - Apr 8th, 2006 at 12:42pm
Print Post Print Post  
Quote:
Thanks Mark.

You have provided some guidelines, but I was wondering about more specifics re Sesame's usage.  For example you provide
Will Sesame use [a-d] or require [AaBbCcDd]?


Sesame allows [a-d] to represent the range from a through d, but is case insensitive. So "^[a-z]" will match any string starting with any letter, including A-Z and a-z.

Quote:
(Hmm, just noticed the typo in the section from the manual).    Any phrases like [:digit:] to represent [0123456789] or [0-9] , or [:space:] to represent blank spaces?
 

I haven't tried them all. Those that I have tried appear to work, but some of the results I am getting appear to be inconsistent with my expectations. I'll have to investigate a bit.

Quote:
I guess I am asking to see an advance of the Sesame manual's section on help for RegEx syntax. 


Pure speculation on my part, but I doubt it will be much more than an equivalence table between some of the common elements between Q&A's syntax and regex, and a pointer to where you can get more information. Part of the point of exposing regex is that it is a well known and common format that has documentation going all the way back to the 1940s. Q&A's syntax is pretty much a closed format.

Quote:
Since you have noted that this will be egrep compatible, then I will use that as a reference.  This could also mean that I can probably use most of any RegEx libraries that I have accumulated.


That would be the hope, that people familiar with regular expressions would be able to use this without having to learn a new search syntax (which by comparison to regex, Q&A syntax is very "new"). In any case, it is likely to be a good subset, if not full implementation of the Posix.2 standard.

Quote:
In addition to using RegEx in XL statements, will there be a RegEx based Search/Replace function for strings?  Or will the existing @Replace just be enhanced to use RegEx?


So far the only places that expose regex are those places where Q&A search syntax is in use - and only some of them. It is not universally exposed. For example, it is not exposed where its exposure would cause an incompatibility with previously defined SBasic functions due to the addition of a necessary syntax choice flag.

It is impossible to determine if regex or Q&A syntax are in use syntactically. For example, "Q.." is a legal in both syntaxes, but means different things. I have considered adding a bracing to the Q&A syntax to indicate the start and end of regex "embedded" within Q&A syntax. But parsing rule changes take a long time to implement, debug, and test.

While regex is a lot stronger than Q&A syntax for string matching. Q&A's syntax is stronger for numeric fields, and gives us simplified conceptual constructs like min and max. Neither give us the boolean expression queries, with parenthesized and / or / not relationships.

If I add constructs to the Q&A syntax in the near future, it is likely that I will be adding localized boolean operators, so it is easier to construct parenthesized queries, either on the form, or in a popup dialog - without resorting to the SBasic programmed query syntax.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #23 - Apr 8th, 2006 at 4:06pm
Print Post Print Post  
It appears that while Posix.2 compliant and Sys V 8 compliant, the implementation is not a full implemenation of egrep's set, and does not support the extended classes (i.e. [:blank:], [:alpha:], etc...) I could add these without too much difficulty (and just did as an experiment), but the performance hit is substantial and there is no trivial way to make them optional.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: Combining Elements to Make Unique Ext. Key
Reply #24 - Apr 8th, 2006 at 4:28pm
Print Post Print Post  
Quote:
but the performance hit is substantial and there is no trivial way to make them optional.


Is the performance hit only when it used or just for implementing or incorporating it causes general reduction in performance even when it is not used?
« Last Edit: Apr 8th, 2006 at 6:55pm by Bharat_Naik »  
Back to top
 
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #25 - Apr 8th, 2006 at 6:48pm
Print Post Print Post  
So it sounds like we may be able to search for a simple situation, for example, where the word "Sesame" starts in the 67th position; or "Cow" is followed by an IP address somewhere in the same line like this? 
Code
Select All
^.{66}[Ss]esame
or
[Cc]ow.*[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3} 


More importantly, I was hoping for search/replace to be used as string manipulation tools vs. just searching. Maybe with options to be turn Case Sensitive On/Off.  Would be good to be able to extract strings that meet RegEx Search specs.

Maybe two functions with optional file lines, that could allow executing against a variable or against a complete file, like

@RegexSearch(source, "RegEx Search Spec",[line]) returns the match as a string or "not found" and
@RegExReplace(source, "RegEx Search Spec", "RegEx Replace Spec",[line]) returns success/fail
  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #26 - Apr 8th, 2006 at 7:06pm
Print Post Print Post  
Quote:
It appears that while Posix.2 compliant and Sys V 8 compliant, the implementation is not a full implemenation of egrep's set, and does not support the extended classes (i.e. [:blank:], [:alpha:], etc...)
That is a good example of why I was asking for documentation.  There is nothing more frustrating then spending time with coding, blaming yourself for syntax errors, etc. and then to contact Tech Support and have them say, "Oh, our version doesn't support that feature".  (I am not accusing Lantica of doing that!).

So again, to be clear, I am hoping the RegEx documentation is not going to say "Posix.2 and Sys V.8 compliant, see egrep manuals."  I am hoping that any exceptions will be detailed. The "e" portion of egrep is for "extended", so your comment that some of the extended classes are not supported is an example of what should be included.  A listing of what is included and what is not included.

Thanks for listening, and more importantly, thanks for starting to include some of the basic RegEx tools for use in Sesame.   Grin Grin

  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #27 - Apr 8th, 2006 at 8:17pm
Print Post Print Post  
Quote:
Is the performance hit only when it used or just for implementing or incorporating it causes general reduction in performance even when it is not used?



I would leave it in if the performance hit was only when such extensions are used. The problem with these "classes" is that they are much like string substitution macros. That means that the search spec string would need to be scanned for their existence, even if they are not being used (to find out if they are being used). If I took some time I could probably implement something that only takes a minimal performance hit, but these "classes" are a convenience and fairly easily replaced with the codes they represent.

For example:

class:                 code:
[:alnum:] = [A-z,0-9]
[:alpha:] = [A-z]

etc...

So, while I may get around to it, I can't really pull it up to the top of my in box. Our best bet would be to get a regex library that is a little more recent. Regex, unfortunately, has a fairly complicated API - involving subcompilation and retained results, etc... So replacing this library is a difficult task and would definitely hold up 2.0.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #28 - Apr 8th, 2006 at 8:41pm
Print Post Print Post  
Quote:
More importantly, I was hoping for search/replace to be used as string manipulation tools vs. just searching. Maybe with options to be turn Case Sensitive On/Off.  Would be good to be able to extract strings that meet RegEx Search specs.

Maybe two functions with optional file lines, that could allow executing against a variable or against a complete file, like

@RegexSearch(source, "RegEx Search Spec",[line]) returns the match as a string or "not found" and
@RegExReplace(source, "RegEx Search Spec", "RegEx Replace Spec",[line]) returns success/fail


These may well be implementable. I'll see what I can do. Its a lot easier to add something new than adapt something existing.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: Combining Elements to Make Unique Ext. Key
Reply #29 - Apr 9th, 2006 at 12:08am
Print Post Print Post  
Quote:
So replacing this library is a difficult task and would definitely hold up 2.0.

Let me be clear about my thoughts on this......

STOP ALL WORK ON THINGS NOT IN 2.0   
GET VERSION 2.0 OUT INSTEAD.


This from one of the strongest requesters of RegEx.
  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
Page Index Toggle Pages: 1 [2] 3 
Send Topic Send Topic Print Print