curl Your RPG Apps with QSHCURL

In this post we’ll take a closer look at how IBM i developers can use the QSHCURL command to easily reach out from CL or RPG and talk to internet-based services and APIs, then consume the resulting data, without a lot of extra effort.

First we’ll provide a short curl intro, and then we’ll look at an example of how to use the PASE-based curl command with an RPG program.

Intro to curl

“Client for URL,” or curl as we know it today, has been a connectivity staple in the open systems world for over 27 years. It lets you grab data for use within your programs using a huge range of protocols and file types—HTTP, HTTPS, APIs, DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, Telnet, TFTP, and so on.

According to Everything curl by curl’s developer, Daniel Stenberg, curl was originally created to automate the fetching of currency exchange rates for IRC (Internet Relay Chat) users. Now curl comes included with the IBM i open source environment (yum install curl).

When you use curl in your RPG apps, you no longer need specialized expertise to interact with internet and other web-based services—whether inside or outside your organization. Imagine being able to utilize curl from a CL or RPG application to take on the duties of downloading website contents, sending and receiving files with FTP or SFTP, sending email messages, or interacting with a web service—such as a UPS or Fedex shipping service—that returns results in XML, JSON and other formats. Combined with an RPG-friendly parser such as YAJL (Yet Another JSON Library), you have all you need to interact with all these services from RPG.

Integrating curl with RPG

Now that you know what curl is and what it can do for you, let’s see how we can apply it from our RPG or CL applications using the QSHCURL CL command.

QSHCURL is a CL command included in my QShell on i utility, QSHONI. QSHONI makes it easy for traditional CL, RPG, and COBOL programs to call Python, PHP, Node, Java, and other QSH/PASE utility programs and directly use their output. QSHONI opens up a whole new world of integrations to open-source apps from RPG, CL, and COBOL.

Previously, I recommended two specific RPG-centric methods of communicating with websites or services from RPG: 1) to use the Db2 HTTP functionality (e.g. HTTPGETCLOB), part of the built-in Db2 database, or 2) HTTPAPI, which is available from Scott Klement.

I found the Db2 HTTP way of communicating to be a little kludgy. And as much as I like HTTPAPI, I like curl even better for future-facing apps because any developer can use it, not just RPG/CL developers.

Moreover, Db2 HTTP functionality and HTTPAPI are limited to web service calls using http and https.

curl does so much more than http and https calls. And it can be used in RPG, CL and even COBOL, as well as with QShell/PASE apps.

So if you want greater flexibility moving forward, it’s worth considering curl for your external communications from the classic IBM i language stack.

curl-ing Your Programs

The following examples show you how to call curl via the QSHCURL CL command from within an RPG app. When curl completes, we will instantly read back the resulting JSON-formatted data returned by curl, and immediately use the results in an RPG application to display the data. This removes the need for your RPG applications to know about any of the plumbing needed to talk to a remote web service. curl does all the work, and it’s a much more standardized method of accessing remote services.

For this sample we’ll do a simple curl call to a public JSON test site named: date.jsontest.com. The site will return a simple single record JSON data packet containing current date/time info that looks like this:

The JSON contains a date, time and milliseconds since epoch (beginning of computer time). (Check out https://jsontest.com to learn more about the JSON test site if you have a chance. There are several sample endpoints for returning JSON data.)

Next we will highlight key sections of the sample RPG programs. The full QSHCURL and RPG samples—JSONTIME.RPGLE and JSONTIME2.RPGLE—are located in my RPG Coding Examples github repository.

The first step will be to call the QSHCURL command to return a JSON packet with current date/time info from data.jsontest.com with a parameter of: –url date.jsontest.com. Internally, the QSHCURL command calls curl --url date.jsontest.com.

Here is the RPG code that retrieves the JSON date and time information:

The QSHCURL command by default will send the JSON result data to STDOUT (Unix-style standard output), but here we specify an IFS file to write the JSON results to: /tmp/jsontime.json. Then we use the popular YAJL library to parse the JSON.

In the data-into example shown below (jsontime.rpgle), we read the IFS file and parse its JSON contents into RPG fields using the YAJLINTO JSON parsing program example from the YAJL library.

After the JSON is parsed, the results are displayed on screen using the DSPLY operation in the RPG sample program.

The second sample program (jsontime2.rpgle) does essentially the same JSON processing after calling QSHCURL, but it does the work a little differently, by parsing the JSON nodes after loading the whole JSON file into memory. We provide this second technique in case you prefer one way or the other. Both RPG samples produce the same result: parsed JSON data.

After the JSON is parsed, the results are displayed on screen using the DSPLY operation in the RPG sample program.

As you can see from the RPG examples, the process of integrating curl to an RPG application using the QSHCURL command is quite simple. QSHCURL handles all the details of the QSH/PASE curl calls, after which we can parse the results using RPG, without any worry about the communications plumbing. As a reminder, curl is not limited to http and https, but can communicate with dozens of other protocols and services as well.

As a final note, QSHCURL is really a curl-specific wrapper around the more generic QSHEXEC command. If you need greater control of the QSH/PASE command line with curl and other apps, you can use the QSHEXEC command instead.

Have fun, and let us know how you’re using curl with your RPG applications.