Using the IS Repository for Data Storage By Ray Moser

---

Introduction

Wouldn’t it be great to store data in the middle of a routine quickly and easily with the confidence to know that it is safe? The webMethods Integration Server (IS) data store provides a such a powerful method to store and retrieve data. The data store (I will refer to it as the Repository) is often misunderstood, mistrusted and underused. This article provides a clear and concise position for using the Repository in your flow code.

---

A Repository Primer

The Repository or data store provides an implementation of a key-value hash table. The main attraction however, is that the data is persisted in permanent file system storage.

Developers should use the Repository to store their global variable data. It also provides a clean way to bridge data between clustered Integration Servers.

What are some of the use cases for this temporary storage? I have used it to store debugging data to avoid the I/O hit of constantly writing to a log. Even though IS writes the data to a "VH" file, the algorithm is super fast -- definitely quicker than calling a logging routine. Since the Repository supports any data type, you can map an entire "debug" record in one swoop that would normally require a sub-flow to map or concatenate all of the data for the normal log routine. If you feel you must have this data in a log (for non-WM personnel to view), then you can perform a pub.storage:keys, and loop through the errors and write in one I/O session.

I have used the repository to store line numbers for custom flat file processing (for record error recording to return the line number back to the producer.)

I currently use the Repository to store a custom profile for a partner server build. Because of the number of different developers involved in projects over time, the code set accumulated multiple property files. Each property file served a different purpose. They required separate and distinct methods to obtain the data. In one case, it was a startup service that initialized values. They are now consolidated in a new profile that is stored in the Repository and the data is callable using common methods.

---

How To Put the Repository to Work

Here are the steps required to begin using the Repository:

Before you can store data, you need a place to put the data. You must create or open a data store by calling pub.storage:registerStore. If the data store doesn’t exist, IS creates it. Once the data store is open you may "Put" or "Get" data by calling either pub.storage:get or pub.storage:put.

To "put" some data into your new store, call pub.storage:put, input the storeName, key and value. To get the same data back, call pub.storage:get, input the storeName and key and your data will be returned.

The following "Supermarket" (store name) table (Keys = Apple through Lime) is representative of a typical key/value storage pairing.

Name Value Apple Red Orange Orange Pineapple Gold Grapes Green Watermelon Red Bananas Yellow Lemon Yellow Lime Green

 

What happens if you forget the keys? Simple! Just call pub.storage:keys. It returns a string list of key names. In the Fruit table above, a call to pub.storage:keys will return all the values in the Key column Apple through Lime.

---

Storing Debug Information in the Repositiory

Another example is stored debugging data. Create a new Repository where storeName = DebuggingData.

In flow code, insert the call to pub.storage:put at each capture point. The keys could be break point IDs. At the end of the processing event (could be a queue of documents), you could kick off an custom error logging routine that calls pub.storage:keys. The result would be a string list of all the keys to the data. Loop through each key, obtain and concatenate the values (assuming field delimiters and record delimiters) and insert into the debug log in one I/O file write transaction. You’ll pick up a little bit of time rather than writing to a log file every time. The time savings isn’t evident until high volume transactions are in place.

---

[1]  2  Next>>

Go Deeper on the Subject: The wMUsers Discussion Forums