In Praise of the Invisible DB2 for z/OS System

I've been working with DB2 for z/OS -- as an IBMer, as a user, as an independent consultant -- for 30 years. I was with IBM when DB2 was introduced to the market. I've met people who were part of the core team at IBM that invented relational database technology. I was privileged to serve for a year as president of the International DB2 Users Group. DB2 has been very good to me, and the work I do as a DB2 for z/OS specialist I do with commitment and passion. And here I am, getting ready to tell you that I hope your DB2 systems will become invisible to a key segment of the user community. Have I gone off the rails? You can read on and decide that for yourself.

To begin, what is that key segment of the user community for which, I hope, DB2 for z/OS will be invisible? Application developers. Why? Simple: the most meaningful indicator of the vitality of a DB2 for z/OS environment is new application workload. Growth, volume-wise, of existing DB2 for z/OS-accessing applications is certainly welcome, but you know, as a person supporting DB2, that your baby is in really good shape when newly-developed applications that interact with DB2 are coming online. If you want this to happen -- and you should -- then you should understand that a vital success factor is having application developers who are favorably inclined towards writing code that accesses DB2 for z/OS-managed data. And THAT depends largely on those developers not having to do anything different, in creating DB2-accessing applications, than they would do in working with another data server. In other words, you want application developers who are working with DB2 to not have to notice that DB2 is the database management system with which they are working.

Why is invisibility, from an application developer's perspective, of major import when it comes to growing new application workload for a DB2 for z/OS system? Several reasons:

• First, you need to realize that a fundamental balance-of-power shift has occurred over the past 10-15 years: developers, once beholden, in many shops, to "systems" people, are now in the driver's seat. As far as I'm concerned, this is as it should be. I believe that the value of IT systems (and the value of the data housed in those systems) is largely dependent on the value of the applications that use those systems (and access that data). Applications bring in revenue, serve customers, schedule deliveries, determine product mix, pay suppliers, manage inventory, etc., etc. If you are a DB2 "systems" person (e.g. a systems programmer or a database administrator), my advice to you is this: view your organization's application developers as your customers, and seek to excel in customer service. If what those folks want is to not have to worry about the particulars of a given database management system when developing an application, deliver that.
• Second (and related to the first point, above), DB2 invisibility, in the sense in which I'm using that phrase, removes a reason for which some developers might be opposed to writing DB2 for z/OS-accessing code. Put yourself in a developer's shoes. Wouldn't your productivity be enhanced if you could write database-accessing code without having to stop and think, "Hmm, for this application, I'm working with DBMS X versus DBMS Y. I know there are some things I need to do differently for DBMS X. Hope I can recall the details of those differences?" Wouldn't it be better if you could just code the same way regardless of the target data server?
• Third, when a DB2 for z/OS system looks to a developer like other data-serving platforms, what becomes more noticeable is the way it acts versus other platforms. Developers might notice that this one data-serving system (which happens to be powered by DB2 for z/OS), for which they code as they do for other systems, always seems to be up, never seems to get hacked, and performs well no matter how much work gets thrown at it. Noticing these characteristics, developers might start expressing a preference for the data associated with their new applications being stored on the platform with the industrial-strength qualities of service. A friend of mine who is a long-time DB2 for z/OS DBA has a good nickname for the system he supports: the "super-server." I can certainly imagine a team of developers making a request along the lines of, "Could we put this application's data on the super-server?" Who wouldn't want that?

OK, so DB2 invisibility is a good thing. How to achieve it? The answer, in a word, is: abstraction. I want to talk here about two especially important forms of abstraction available for DB2 for z/OS systems -- one with which you are likely familiar, and one which may be news to you.

The familiar form of DBMS abstraction to which I referred above is the kind that makes a particular relational database management system (like DB2 for z/OS) look, to an application program, like a generic relational DBMS. In my experience, the two most common of these DBMS-abstracting interfaces are JDBC (Java Database Connectivity) and ODBC (Open Database Connectivity). ADO.NET is another example. These interfaces are enabled by drivers. In the case of DB2 for z/OS, JDBC and ODBC (and ADO.NET) drivers are provided, for network-attached applications (i.e., applications that access a target data server through a TCP/IP connection), via the IBM Data Server Driver (or DB2 Connect, though the former is recommended, as I pointed out in an entry posted to this blog a couple of months ago). IBM also provides, with DB2, JDBC and ODBC drivers that can be used by local-to-DB2 programs (i.e., programs running in the same z/OS system as the target DB2 for z/OS data server).

Lots and lots of applications, at lots and lots of sites, have been accessing DB2 for z/OS systems for years using JDBC, ODBC, and other interfaces that make DB2 look like a generic relational database management system. More recently, a growing number of developers have indicated a preference for a still-higher level of abstraction with regard to data server interaction. To these developers, using a DBMS-abstracting data interface such as JDBC or ODBC is less than ideal, because such interfaces indicate that the target data-serving system is a relational database management system. While it's true that the use of JDBC or ODBC (or a similar interface) means that a programmer does not have to be aware of the particular relational database management system being accessed, many developers feel -- and they have a point -- that even the form of a target data-serving system (one form being a relational DBMS) is something they should not have to know. From their standpoint, the form of a target data-serving system is a matter of "plumbing" -- and plumbing is not their concern. These developers want to access data-related services (retrieve data, create data, update data, delete data) as just that -- services. When data-as-a-service (DaaS) capability is available to a developer, the nature of what is on the other side of a service request -- a relational DBMS (like DB2), a hierarchical DBMS (like IBM's IMS), a Hadoop-managed data store, a file system -- is not consequential.

What is consequential to developers who want data-as-a-service capability is consistency and ease of use, and those concerns have much to do with the rapid rise of REST as a means of invoking programmatically-consumable services. REST -- short for Representational State Transfer -- is a lightweight protocol through which services can be invoked using HTTP verbs such as GET and PUT (services accessed in this way can be referred to as RESTful services). Adding to the appeal of REST is the use of JSON, or JavaScript Object Notation -- an intuitive, easy-to-interpret data format -- for the data "payloads" (input and output) associated with REST calls.

REST enables client-side programmers to be very productive and agile in assembling services to create applications, and that likely has much to do with REST being widely used for applications that access a server from a mobile device front-end and/or from cloud-based application servers. But DON'T think that the cloud and mobile application worlds are the only ones to which RESTful services are applicable. There are plenty of developers who want to use REST API calls to build in-house applications that may have nothing to do mobile devices or clouds, for the reasons previously mentioned: productivity and agility. Mobile and cloud application developers aren't the only ones who don't want to deal with IT infrastructure plumbing.

DB2 for z/OS moved into the realm of RESTful services in a big way with the native REST interface built into DB2 12 (and retrofitted to DB2 11 via the fix for APAR PI66828). That DB2 REST API can be invoked directly from a client, or by way of IBM z/OS Connect (as mentioned in my blog entry referenced previously in this post). When z/OS Connect is in the picture, not only DB2 services, but CICS, IMS, WebSphere Application Server for z/OS, and batch services, as well, can be accessed via REST calls from clients, with data payloads sent back and forth in JSON format. z/OS Connect also provides useful tooling that facilitates discovery (by client-side developers) and management of z/OS-based RESTful services. Whether the DB2 REST API is accessed directly or through z/OS Connect, what it does is enable you to make a single SQL statement -- which could be a call to a stored procedure or maybe a query -- invoke-able via a REST call with a JSON data payload. Some very helpful information on creating and using DB2 RESTful services can be found in an IBM developerWorks article by Andrew Mattingly, in examples posted to developerWorks by IBM DB2 for z/OS developer Jane Man, and in the DB2 for z/OS IBM Knowledge Center. Keep in mind that because the DB2 REST API is a function of the DB2 for z/OS Distributed Data Facility (DDF), SQL statements and routines invoked via that interface are eligible for zIIP engine offload (up to 60%) just as are any other SQL statements and routines that execute under DDF tasks.

And lest you think that DB2 invisibility is applicable only to "operational" applications...


An invisibility provider for analytical DB2 for z/OS workloads

Data scientists, like application developers, can be more productive when they don't have to think about the particulars of the data server with which they are working. As interfaces such as JDBC and ODBC and REST provide much-appreciated abstraction of data server "plumbing" to application developers, so, too, for data scientists, does an analytics-oriented software technology called Spark.

Spark, produced by the Apache Software Foundation, is an open-source framework for data analytics. It can run natively under z/OS (and on zIIP engines, to boot) because, from an executable standpoint, it's Java byte code (Spark is written in the Scala programming language, which when compiled executes as Java byte code). The IBM z/OS Platform for Apache Spark provides connectors that enable Spark to ingest data from z/OS-based sources, such as VSAM, that do not have a JDBC interface. Add that to Spark's native ability to access data sources with a JDBC interface, such as DB2 and IMS, and data in Hadoop-managed data stores, and you have, with Spark under z/OS, some very interesting possibilities for federated data analytics (Spark builds in-memory structures from data it reads from one or more sources, and subsequent access to those in-memory data structures can deliver high-performance analytical processing).

If you go to one of your organization's data scientists and say, "There is some data on the mainframe that could probably yield for you some valuable insights," you might get, in response, "Thanks, but I don't know anything about mainframe systems." If, on the other hand, you lead with, "We've got some really important data on one of our systems, and we're running Spark on that system, and you could analyze that data using R and uncover some insights that would be very helpful to the company," you might well get, "Okay!" (the R language is very popular with data scientists, and there is an R interface to Spark); so, platform abstraction can be a big plus for analytical, as well as operational, DB2 for z/OS workloads.


And on more thing...

(As Steve Jobs used to like to say)

I generally can't leave a discussion about DB2 for z/OS invisibility (from the application development perspective) without bringing up DB2 row-level locking. I'm often left shaking my head (and have been for over 20 years) over how reluctant many DB2 for z/OS DBAs are to allow the use of row-level (versus the default page-level) locking granularity. It's as though DBAs -- especially verteran DBAs -- think that they will have to turn in their DB2 for z/OS Club cards if they are caught with table spaces defined with LOCKSIZE(ROW). Recall what I pointed out near the beginning of this blog entry: the really meaningful measure of the vitality of the data-serving platform you support is growth in the form of new application workloads. Getting that type of workload growth means presenting a developer-friendly environment and attitude. Platform invisibility is very helpful in this regard, and other relational DBMSs that protect data integrity with locks (including DB2 for Linux, UNIX, and Windows) do so, by default, with row-level locking granularity. Telling a development team with programs that are experiencing lock contention issues with DB2 for z/OS page-level locking in effect that the problem is their code is not going to win you friends, and you want to be friends with your organization's developers. Yes, you should use page-level locking where you can (and that is often going to be for the large majority of your table spaces), but selectively using row-level locking and incurring what is likely to be a relatively small amount of additional CPU cost in return for growth in new-application workload is a good trade in my book. I have more to say about this in an entry I posted to this blog a couple of years ago.

And that's that. Yeah, DB2 for z/OS is your baby, and you like it when people tell you how beautiful your baby is, but to keep that baby healthy and growing it often pays to make the little darling invisible to application developers (and data scientists). When those developers talk about that whatever-it-is data-serving platform that's as solid as a rock, you can smile and think to yourself, "That's my baby!"