We were also embarking on a relatively new technology; namely: Microsoft's System Management Server (SMS) version 1.1. This ESD (Electronic Software Distribution) system was to manage the 4,000+ client base.
We had been using Novell login scripts to push down "package" installations utilizing chained batch programs. The SMS solution would provide a 'Central Management' console and a offer a more complete packaging, distribution, monitoring and support solution.
Security on the client was not an issue with the Windows 3.1 Operating System but we knew it would become an issue with the upcoming planned implementation of Microsoft Windows NT 4.0 and/or Windows 2000 Professional.
Near the very end of the UAT and QA testing we ran into a severe road block in the critical path of the Client/Server project. Intermittent Oracle errors were occurring after we had already stepped through and resolved many such Oracle errors before. However, this set of errors were different and were a real problem because we were less than 1 week away from performing the 'Turn Key' migration from the Mainframe applications and to the Client/Server environment .
We could not consistently repeat the errors, nor could we pinpoint what had changed (even after studying all the related Change Controls). I went back into the Computer Support test lab and worked with several Teams (Application Developers, Database Administrators, Midrange Server Group and Computer Support) to resolve the problem. Soon we discovered that it was the middleware (Oracle's SQL*Net) that seemed to be responsible for the errors.
This was now my task to resolve. I was solely accountable for the solution. I was also under great pressure from the CIO and the Board of Directors to resolve the problem. The reasons for the pressure are obvious but it also would incur great unanticipated expenses to continue and renew the costly leases for the IBM Mainframe environment (hardware and operating environments).
After several meetings with Senior Management, I was back in the lab and began to utilize the Oracle logging and tracing utilities that were installed with the middleware (SQL*Net) to attempt to pinpoint the issue. Continuous calls and onsite support from Oracle and Microsoft were insightful but did not solve the issue.
Quickly I discovered that when I activated logging and tracing to debug the middleware components the errors ceased to occur. The log and trace files were also clear of error and failures. An end user (a great guy who went by the nickname of "Doc"; partially due to the fact that he was very computer literate) quickly coined my new nickname of "The Professor".
The Bottom Line:
We determined that the problem was the new client hardware being rolled into the test Client/Server environment, i.e. the Pentium PCs.
The majority of the Power Builder application code and infrastructure testing was performed using the 486 base (early and most of the way into the testing). Introduction of the new client platform was causing the Oracle and midrange environment to "overload" and fail on SQL requests from the faster client based hardware. This was especially seen when more and more Pentiums were introduced and actively connected to the new Client/Server environment through the applications.
We determined that there was insufficient time to dig into altering and retesting the application code, nor was there time to somehow adjust the Oracle / HP-UX Midrange environment; too much testing had already been performed to this point to go back to the drawing board in that arena.
Therefore, I recommended that we leave logging and tracing on for the production roll out. By adjusting the oracle.ini and sqlnet.ora, etc. configurations on the Novell servers we accomplished our goal on time and under extended budget.
All log files were written to a common area on the Novell Servers so as to not cause 'bandwidth' issues with the client. We cleared the log and trace files off the Novell Servers continuously; since they were meaningless and just taking space.
By the way, we decided to include client PC hardware changes into the Change Management process from that point forward.
Why angry dinosaurs?
Well, the new plan with respect to populating real time manufacturing production data remained with the Mainframe environment. Instead of a goal to migrate off of 90% of the mainframe population, the corporation decided to maintain 30% of the Mainframe space.
The Mainframes were back in the equation and they were angry (i.e. the people that maintained them were angry). We still needed them, and they knew it from the start.
Bleeding edge? Yep.
NEXT, we had to migrate from Novell to Microsoft ; and implement Microsoft Systems Mangement Server. See the next story.
The End.