We are in business to help retailers manage inventory, using whatever POS (Point-of-Sale) system they prefer. You’ve read a lot here about some of the great ways we help businesses manage inventory, which range from comprehensive inventory management, to sophisticated COG (Cost-of-Goods) calculations (think: multi-currency), to the new stuff we are doing in Analytics. It all starts, though, with our interface to the POS’s that our customers choose to use (and some use multiple types, which is fine with us). This is the history of one of those interfaces, and how DataWorks goes the extra mile to make those interfaces happen, and happen reliably.
RMS: Here We Go Again
The story begins with a request for us to interface with Microsoft’s RMS POS product. We had done work on an interface to POS 2009, the Microsoft product that replaced RMS, but the customer wanted to use RMS because the POS vendor (New West Technologies) has a mobile device made to run on RMS. Since the capabilities the customer needed were also in RMS, we repurposed the interface to run with RMS. Since neither RMS nor POS2009 has an API (standard way of interacting with a program) capable of supporting a POS interface (RMS has none, POS 2009 has one that, let us say, exhibits several design insufficiencies). Anyway, here we were with RMS and no API for us to use: we would have to create both sides of the interface.
Well, we’ve been doing this kind of thing for 20+ years now, so how hard could it be? And in fact, it was quite easy: the database for RMS (a product which Microsoft bought from another company around 1994) is well designed, and we were able to do so in record time. The challenge wasn’t in getting sales data, nor in putting inventory data in: it was in communications.
We have done many kinds of interfaces with POS vendors, but we had no model for doing it entirely on our own: we took this as an opportunity to make something that was easy to deploy, would satisfy Enterprise security needs, was reliable, and could easily be re-purposed. The result was SmartSpoke, which sits on the customer site and uses FTPS (secure FTP) to communicate back to an FTPS server in our co-lo. So far so good: and then there are the pickles (shorthand in our R&D team for the unexpected that comes after you are sure you are done <s>).
It Would Have Been So Easy
The RMS POS can be deployed in more than one way. Well, we know that now… When we designed SmartSpoke, we knew of one way: a SQL Server instance hit by the registers, however many there are in the store or site. The SmartSpoke would consume very few resources, running as a Windows service, and so could co-exist happily on the same box as the SQL Server, which it would also use for its own configuration and logging data. We would be able to get on the SQL Server machine in one manner or another while the Registers were running, in order to handle the inevitable issues that arise when doing interfaces. What issues, you say? Why can’t they all be known ahead-of-time? Well, there are these pickles, you see… We produced a diagram of the system, which we gave to all involved with our first project.
Then we had our first implementation meeting, where we found out a) that each register would have it’s own SQL Server (Express); and that b) the complications of maintaining a server (this was an Enterprise solution), at each location, was not feasible. With stores scattered everywhere, far from corporate headquarters and the IT staff needed to monitor and maintain servers according to the corporation’s standards, the level of effort would be unacceptable, unless we really, really had to have a separate server.
Did I mention that our business is to help businesses manage inventory? Well, giving a business another headache to deal with isn’t a great way to help, so we, with the aid of our POS vendor project partner (New West), tested putting the SmartSpoke services on the same machine as the POS, which also had SQL Express running on it. Would the POS be able to proceed as normal with the SmartSpoke loaded on the same machine? Lo and behold: it worked! We were home free! Except for those darn pickles, wouldn’t you know.
Did I Mention Helping?
The first complication we ran into was the inability to get on the interface when the POS was in use. We report back home (via email) when sales are missing for a particular date. Support then looks into what’s going on. But they couldn’t check the interface when the POS was in use, and so all the checking had to be done after the stores closed or before they opened, which of course was mostly outside our normal business hours. We had no other option than to do any checking, and fixing, of the interface at night or early in the morning. No extra charge, we just did it because it needed to be done. In the meantime, as CTO I began searching for options. But another pickle popped up before we could test and put a solution in place, and that one needed to be solved quickly.
So Much For Testing
You might have thought we didn’t test any of this, which I assure you is not the case: we tested everything we knew to test, based upon the best possible advice on how the system would function. For example, it was the understanding of everyone involved that where there was more than one register, one would be the master, and any other “lanes” (as they are known in retail) would be slaves, connected to the same database. With the assistance of Casey Hansen, VP of Technical Operations at New West, we tested that, and everything worked smoothly. We forced the slave register to start its own sales batch, on its own SQL Express instance, by taking the Master offline. When the Master reconnected, the slave created a new batch on the master POS, put its stored-up data in that batch, closed that batch, and opened a new one on the master, which took its sales from that point on. As a technologist I had a big smile on my face: that’s the way things are supposed to work!
The Pickle, in this case, turned out to be a handheld POS terminal which, unlike the slave registers, created its own batch and kept it open until the user decided to close it. Why did this matter? It’s all about duplicate sales: we cannot let them get into our system. We thought we had the ideal setup, given that the Master Register only held one batch open at a time: if we tracked the last batch number processed, we would know the next one to process. So we adjusted the interface to work off “last datetime,” instead (which we had avoided due to improbable but possible problems with datetime changes on the host computer), and then returned to the issue of finding a way to check on/adjust the interface while the POS was in use.
We Get Help From Others, Too
While working with the firewall engineer for one of the first stores where we installed the RMS interface, a very helpful engineer from Vendorsafe (they specialize in helping businesses become and stay PCI-Compliant – credit cards over the internet and the like) mentioned that other Vendorsafe clients were using the built-in VNC server in VMWare Server and Workstation to connect, and that combined with a very inexpensive SSL VPN they provide, this would be acceptable in maintaining PCI compliance. We would still be running on the POS register, but in our own OS instance inside VMWare. Using VMPlayer, VMWare workstation’s little brother, we would minimize memory utilization (fewer features VMPlayer should translate to less memory used) and minimize cost to our customer.
I quickly put together a test instance and it worked fine. Using the VMWare SDK (Software Development Kit – tools to do non-standard things with a piece of software) I could get the VMPlayer to boot up in “headless” (not visible to the user of the machine) mode. Having gotten permission from the customer to go ahead with testing, I made arrangements with Casey at New West to do the testing. Of course I wanted to get the latest version of VMPlayer installed there, since New West clones the versions on which we test to install at sites. Only one problem: the new VMPlayer version didn’t work with VNC when I installed it at New West.
A bit of research later, accompanied by some mutterings about the darn pickles, and with help from VMWare employee blogs, among others, I found out that a) VMPlayer has all the same API as VMWorkstation, but b) in the latest version, for reasons that were muddled (that’s what engineers do when a decision with which they disagree is made by Marketing), had set the flag off for VNC to work in VMPlayer, and that c) they might turn it back on in the future. The version I had initially tested had been the last one with VNC still working. Given the choices, I did what I usually do in those kinds of situations – more research. Well, it turns out that flag enabling VNC was at a particular place in the exe, and that with a hex editor … well, you get the idea. And if there is ever any objection, I could always go back to the earlier version. In the meantime, this was working great. We threw everything we could at it, myself inside the VMPlayer instance with the SmartSpoke version pushing inventory and taking in sales, at the same time I had the SmartSpoke Manger running, doing things in the UI that taxed the SQL Server instance, which was still on the host computer. Meanwhile, Casey pumped sales into the POS as quickly as he could. The result: no noticeable difference on the POS side. Boy, we had it made now! VendorSafe provided the VPN, which works like a charm, VNC connected as fast as it does on my work network, problem solved. You know what’s coming next…
It Should Have Worked…
Casey from New West put the .bat file starting VMWare in the Startup folder: because the machines were kept on at the various sites over night at the POS login, ready for the next day, VMWare and therefore the SmartSpoke Interface would always be running. Then we got reports of no sales: what??? We found some instances where the POS was at the OS logon, which would account for VMWare not running. But we also found instances where the POS was running, and VMWare was not. Inconceivable! (Another R&D team iconic expression – from The Princess Bride, if it seems familiar <s&ft; .) Casey thought it might be the DOS window that ran when VMWare started up: perhaps the user saw it, and clicked it shut? We had no knowledge that was the case, but he put in a shortcut to the .bat file, with a Minimized window setting, to see if that made a difference. It didn’t.
So, a couple of days ago I spent a few hours creating, testing and installing a service that runs when the OS starts, before logon, and starts VMWare. I think we’ve got it now … but I’m not proclaiming success. Rather, I’m proclaiming that we put in place a solution that appears to be working as intended.
This Is The World As It Is
When I was studying psychology, a therapy supervisor told me “don’t be afraid to make a mistake. You will. It’s what you do with the mistakes that counts.” We know at DataWorks that we are no more, nor less, capable of creating foolproof products than any other organization. Our strength is in the attitude about what we do with our mistakes: we find an answer quickly, and put in place. It’s our ability to respond in an agile manner that allows us to give our users what they need in Retail Inventory Management. Moving quickly as we do makes for an exciting place to work. It also makes for some great feelings at the end of the day, knowing that we’re fulfilling our goal of helping our users do their jobs better and more easily.