EDIviaSFTPfor
Linux
a Client/Server EDI over SFTP Client system for Linux
EDIviaSFTPforLinux was designed as an SFTP client to connect a mixed Linux / AIX LAN to IBM Information Systems SFTP server for EDI exchange. The system consists of:
The system has been implemented and is known to run on RedHat Linux ES3, ES4, 7.2 and 9.0. It should run on other Linux systems. The client is known to run on AIX 4.3. It should run on other versions of AIX.
Except for one utility, the entire system is implemented in the Python programming language. It requires the Python version 2.4.2 or higher. Python can be obtained from www.python.org.
The server [serverEDI] runs as a daemon on a host which is allowed direct or indirect Internet access. [It is probably OK to run on a firewall system] The client [clientEDI] runs on any host on the LAN. It connects on a mutually agreed upon TCP socket to enqueue EDI for transmission and dequeue received EDI. The server does nothing unless commanded, so connections to the external EDI host [e.g. the IBM Information Exchange SFTP server] to transmit and receive queued up EDI is controlled by the client. This makes transmission and receipt scheduling available for local control.
Queues are implemented as simple directories on the local server host. The server manages contention by a simple lock file scheme.
All EDI data which is transmitted is archived on the server host. Similarly, as received EDI data dequeued by local clients, it is also archived as well. As a feature, previously transmitted EDI may be re-queued for repeat transmission to take care of the possibility that the receiving party has a reception failure while avoiding recreating the EDI data from scratch.
Access to the server requires authentication using a simple userid/password protocol. No passwords need be stored on the server host; the password file contains only userid’s and MD5 digests of passwords. The password database is maintained by a separate password editing programming [passwordEDI.py].
Security is controlled by executing all processes as a separate, non-login user [ediuser]. The server, client, and password scripts are owned by ediuser and reside in ediuser’s home directory. Permissions on this directory are set to the tightest permissions possible and still have the software execute.
The actual scripts serverEDI.py and clientEDI.py are buried in the home directory of ediuser – which is read/write/search only by ediuser. The scripts and their direct parent directory are read-only to help avoid adulteration. Execution is performed by a C program which performs a collection of security checks and then executes each script in a restricted environment using the absolute path to the Python interpreter.
All EDI activity is logged in a log files. Each log entry includes the process id of the server instance making the entry; the server is implemented as a forking server, where each client connection gets a separate server process. The userid of the connecting client is logged upon authentication attempt. This allows tracking activity to at least client host granularity by using different userid’s for different client hosts.
All EDI queues, logs, and the password database are placed in a single directory tree which is owned and accessibly only to ediuser.
Download from EDIviaSFTPforLinux Tar File – Latest version is 0.9.4.b as of 2/21/06
The system is packaged in a gzip’ed tar file. Installation is controlled by GNU make. The basic steps to take one each host on which the server or client reside are:
1. Obtain and install Python 2.4.2 according to the directions on the box. This version of Python can be installed anywhere and does not need to be on the default system path. This avoids version conflicts with system software. The install script will search the system and find the version it needs.
2. create a non-login account named ediuser with a unique, single user group.
3. as root, make a build directory and ‘cd’ to it. Untar the distribution.
4. Obtain the SSL certificate from you vendor. Convert them to PEM files using openssl as described below. Make them root-only accessible. Then, create a file named ‘Makefile.certs’ so that the install system can find them. Makefile.certs contains only two variable definitions:
a. KEYS = <path to plain text Key PEM file>
b. CERTS = <path to plain text Certificate PEM file>
5. do an initial install by typing either:
a. ‘make install’ for a client install
b. ‘make INSTALL_MODE=server install’ for a server install
c. ‘make INSTALL_MODE=both install’ for both client and server install
6. This will set up the basic file structure and install all the files.
7. Edit the configuration file /usr/local/etc/EDI.ini. Several parameters must be defined. Define SERVER_HOST, SERVER_PORT, CLIENT_USERID, and CLIENT_PASSWORD on each client host. Define SERVER_HOST, SERVER_PORT, ACCOUNT_USER, PASSWORD, KEY_FILE, CERT_FILE, and FTP_HOST on each server host.
8. Build the password database by su’ing to ediuser and running the password utility passwordEDI.py.
9. Enter a crontab entry which periodically starts /usr/local/bin/serverEDI. The server will check to see if it already running and will terminate if it already is.
10. Re-install the software again using the same make command. This is basically a precaution in case something didn’t work properly during the first install prior to editing EDI.ini. It may not be necessary, but it doesn’t hurt.
11. Start the server and test the connection from a client.
After the installation, three programs will have been installed in /usr/local/bin:
The EDI directory tree will be created in /usr/local/etc/edi-root.
The configurationfile EDI.ini will be created in /usr/local/etc.
Various files and scripts will be installed in the home directory of ediuser.
Server operation is pretty simple. Create a crontab entry of the form:
0,15,30,45 * * * * /usr/local/bin/serverEDI
This will invoke the server every 15 minutes. The server will check for the existence of the file ‘/var/run/serverEDI/serverEDI.pid’. If it exists, it will contain the PID of the last running server. If that server is still running, then the newly started server exits. Otherwise, it continues to run.
The server is a forking server
so that each time a client connects, it spawns a new process. The server was not written with any
consideration for threading issues.
Client operations are similarly simple.
The client logs events in /usr/local/etc/logs/edi-log. It simultaneously logs ‘info’ level and higher events on stderr. Listings of queues are written on stdout so that that information can be separated from log entries.
Client operations fall into four groups:
· Moving EDI files to and from the local server
· Commanding the server to transmit and receive EDI from the remote SFTP server
· Listing the contents of queues and archives.
· Requeueing previously transmitted files for transmission.
clientEDI enqueue <file(s)> copies the named files to the transmit queue on the local server. The log contains entries for each file copied.
clientEDI dequeue [file pattern] copies files matching ‘file pattern’ to the local host. The files will be deposited in the current working directory [unless changed by the –dir option] which must be group writable by the invoking user.
File names are constructed using data from the ISA or UNB records of the EDI data. The file name is always of the form:
<account>-<mailbox>-<control number>-<timestamp>.<suffix>
Where <account> is the recipient account; <mailbox> is the EDI mailbox; <control number> is taken from the ISA or UNB record; <timestamp> is of the form YYYY-MM-DD+HH:mm:SS; and suffix is a number which is normally 0.
The file pattern uses usual UNIX globbing rules. That, combined with the file name structure, allows distinct client hosts and processes to be selective in retrieving EDI data.
clientEDI transmit and clientEDI receive cause the server to connect to the remote server and perform transmit queued EDI or receive and queue EDI waiting on the remote server.
When receiving, all EDI queued on the remote server is pulled down.
Results of both receipts and transmission of each file is logged in both the log file and to stderr. The log entry for transmitted files includes the receipt acknowledgement by the remote SFTP server.
clientEDI list <token> lists the file names for all files in one of four directories on the server. The following values of <token> are defined:
clientEDI requeue <file name> copies the named file from the Transmit Archive to the Transmit Queue. It will be transmitted the next time the server is commanded to transmit data.
The password database is administered by the program passwordEDI.py which resides in the home directory of ediuser.
passwordEDI.py executes the following commands:
passwordEDI.py add <userid> <password> adds the userid and md5 hash of the password to the password file.
passwordEDI.py
passwordEDI.py chg <userid> <new-password> changes the password of <userid>, unless <userid> is not in the password file.
passwordEDI.py chk <userid> <password> tests to see if <password> is correct for <userid> and prints a nice string announcing the result.
passwordEDI.py list lists all the useridâs in the password file in ascending ASCII order.
passwordEDI.py properties lists all the useridâs and properties in the password file in ascending ASCII order by userid.
passwordEDI.py setp <userid> <property> sets the property value of <userid> to <property>
passwordEDI.py getp <userid> retrieves the property value for <userid>..
There shouldn’t be much required in the way of maintenance. Log files are size limited and are supposed to roll over when they exceed the limit. Only the previous five or so are maintained.
Archive files are all compressed, but the number of entries grows without bound. The system does not currently contain any means of limiting the number of stored files.
Bug reports and suggestions should be sent to Mike Howard mike@clove.com.