Hot Topic (More than 10 Replies) EBCDIC Problem (Read 7524 times)
BWETTLAUFER
Full Member
***
Offline



Posts: 216
Location: Cambridge, Ontario
Joined: Apr 9th, 2010
EBCDIC Problem
Jul 19th, 2012 at 8:07pm
Print Post Print Post  
Hey all,

I have an issue ... I have a client giving me a weekly packed decimal EBCDIC file via FTP.  It's a fixed length file with specific positions being EBCDIC, and specific positions being packed decimal/Comp-3 format.

Currently I am translating it by hand with Notepad++ and plugins, translating the Comp-3 to EBCDIC, and then translating the whole thing to ASCII, but I'm thinking there has to be a better way.

Can anyone suggest a solution I could use to automate my translation? 

Thanks!
  
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: EBCDIC Problem
Reply #1 - Jul 19th, 2012 at 9:03pm
Print Post Print Post  
At least its not actually still on punch cards...

Are you doing business with a computer history museum?

Ok. Enough smart-alec remarks, do you want me to look into this for you? Its probably not hard to write a custom program to manage this. Given the age of this problem, its not too likely you'll find a canned utility (but maybe).
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
BWETTLAUFER
Full Member
***
Offline



Posts: 216
Location: Cambridge, Ontario
Joined: Apr 9th, 2010
Re: EBCDIC Problem
Reply #2 - Jul 19th, 2012 at 9:06pm
Print Post Print Post  
I'd love some help!

If you like, I can send you a sample of the EBCDIC record I am working with. 

I found some canned utilities, but they require me to have a mainframe handy, learn COBOL, or drive a DeLorean...
  
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: EBCDIC Problem
Reply #3 - Jul 19th, 2012 at 9:12pm
Print Post Print Post  
I'll take a look at it. Send it to support, Ray will forward it to me.

Make sure you include any details I'd need, like where different formats begin and end.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
BWETTLAUFER
Full Member
***
Offline



Posts: 216
Location: Cambridge, Ontario
Joined: Apr 9th, 2010
Re: EBCDIC Problem
Reply #4 - Jul 20th, 2012 at 6:05pm
Print Post Print Post  
Thank you sir ... the email has been sent!
  
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: EBCDIC Problem
Reply #5 - Jul 20th, 2012 at 9:02pm
Print Post Print Post  
I looked at your format. Converting the EBCDIC to ASCII is trivial. Every UNIX based OS comes with a utility that can do this. The difficulty comes with translating the packed decimals... well not so much the translating, that is also pretty easy and I could write script or program that does them one at a time, in less than an hour - but in finding them and inserting them back into the (already mostly translated) document in the right places. I could write a program that uses the format document to read in and translate, spitting out the final document in the same fields, in a few hours.

Basically, the difficulty is in the variation introduced by having 16 possible record types.

Were I to take this on, I would need to have a couple questions answered: does the after document have ASCII or actual binary numerics in place of the packed numerics in the original? I'd need to be able to check my work. I'd need a before and after of a small set of packed numerics.

And, given that this is likely to run into several hours or work, I would need to be able to bill your company for the time.

I may be cheaper to hire in the short run. But, I suspect that it would save a tremendous amount of money in the long run to get this company off of such archaic formats, languages, and hardware.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
BWETTLAUFER
Full Member
***
Offline



Posts: 216
Location: Cambridge, Ontario
Joined: Apr 9th, 2010
Re: EBCDIC Problem
Reply #6 - Jul 24th, 2012 at 1:41am
Print Post Print Post  
Hi Mark,

The layout doc is misleading ... it gives the option of 16 different layouts, but only one is really used.  The attached before and after sets out the only format we receive.

I'd be fine with you billing my company ... it would be better in the long run, for certain!

It would be great to have my client convert to some sort of data format common to the last 50 years, but I'm not going to win that battle.

If you want to give me a call or email me directly, I can probably answer any questions you have about the output, since I've been manually translating the weekly data files for almost two years now. Smiley
  
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: EBCDIC Problem
Reply #7 - Jul 24th, 2012 at 1:05pm
Print Post Print Post  
Are you positive that only one of the 16 formats will ever be used?

How are you translating the packed binary by hand? What format is the result of the packed binary? An ASCII representation of a numeric? A binary numeric?

I will get your email from Ray.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
BWETTLAUFER
Full Member
***
Offline



Posts: 216
Location: Cambridge, Ontario
Joined: Apr 9th, 2010
Re: EBCDIC Problem
Reply #8 - Jul 24th, 2012 at 5:47pm
Print Post Print Post  
Hi Mark,

In the last 2 years at this company, and the last 10 years at the previous company, this client has only ever given me one layout format.  I'm pretty sure it's good.

I'm currently using Notepad++ with a Comp-3 translation plugin, translating by hand all non EBCDIC values to EBCDIC, and then the EBCDIC to ASCII. 

My email is bwettlaufer (at) kingstondc (dot) com

Thanks!
  
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: EBCDIC Problem
Reply #9 - Jul 24th, 2012 at 6:17pm
Print Post Print Post  
When you say "non-EBCDIC" values, do you mean the packed binary numbers?

Can you send me a small before and after (no more than two records)?

Which record type are they all?

It may be possible to do what you are doing by automating notepad++. Have you looked into that?
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
BWETTLAUFER
Full Member
***
Offline



Posts: 216
Location: Cambridge, Ontario
Joined: Apr 9th, 2010
Re: EBCDIC Problem
Reply #10 - Jul 31st, 2012 at 3:17am
Print Post Print Post  
Well, I've tried playing with macros in Notepad++ with some limited success ... I'm going to keep trying ...
  
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: EBCDIC Problem
Reply #11 - Jul 31st, 2012 at 12:40pm
Print Post Print Post  
It looks like notepad++ has support for several scripting languages, most particularly Python.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged