April 20, 2004 Update as of 6:45pm
As of 6:45pm today the Student Information System has been fully restored and will remain available until its normal downtime of 10pm.
We sincerely apologize for the inconvenience caused by today’s problems.
April 20, 2004 Update as of 5:30pm
As of 5:30pm today AIS and SIS teams are still working with IBM to resolve the problems with the mainframe Student Information System reported in this morning's announcement. Our teams will continue to work through the evening until the system is fully restored.
Please continue to use infoline@columbia.edu to contact the SIS Support team with any concerns and/or questions
April 20, 2004 Update
As of 2:40pm today AIS and SIS teams are still working to resolve the
problems with the mainframe Student Information System reported in this
morning's announcement. IBM has been called in and is currently
consulting with AIS to find a solution.
Please continue to use infoline@columbia.edu
to contact the SIS Support team with any concerns and/or questions.
April 20, 2004
The mainframe Student Information System (SIS) is being brought down immediately, and will remain unavailable until mid-afternoon. We apologize for the serious inconvenience this will cause, especially as we are in a registration period.
A serious problem was encountered during last night’s SIS batch processing cycle that caused a data file to run out of space, and the cascade effect also delayed the updating of the Report Server and SSOL databases. Our colleagues at AIS are working as quickly as possible to diagnose and resolve the problem.
Both the Report Server and SSOL have now been updated, and will remain available even while SIS is down. However, the registration application in SSOL will *not* be available until SIS is fully restored.
Please watch this webpage http://www.columbia.edu/cu/sis/announcements.html for updates on this situation and contact us at infoline@columbia.edu with any concerns or questions about SIS, SSOL, Report Server.
August 16, 2003
This is to inform you that SIS, SSOL and SIS Desktop
Reporting are all recovered and up-to-date since the
blackout.
When power was lost on Thursday at approximately 4:10pm,
backup generators allowed enough time for our colleagues at
AIS to perform a smooth and controlled system shutdown.
However, if you were in the middle of entering data into
SIS and did not perform a 'save' function before the power
was lost, it is likely that specific data is gone. In that
case, please review your work from that time and/or contact
our staff for assistance. No other data was lost during the
shutdown. Overnight processing was not performed on
Thursday night, and no systems were available on Friday. By
Friday evening, AIS had the mainframe back in business, and
overnight processing was run for SIS on Friday night. The
Report Server and SSOL databases were then updated, so all
applications are now current.
If you have any further questions, please send e-mail to
infoline@columbia.edu and we will address your inquiries
and concerns on Monday.
I hope you all got through the blackout with little
trouble, and enjoy a good weekend.
March 26, 2003
During the weekend of April 5th and 6th, our colleagues at AIS will upgrade
the University mainframe computer's operating platform (CICS) to the newest
available version. As a result, the Student Information System (SIS) and
all SIS-related mainframe services (including overnight requests and FOCUS
reporting) will be unavailable on Saturday, April 5 and Sunday, April 6.
Please do not attempt to use SIS during this time; even if you succeed in
logging into SIS you will be working in a test environment, and any changes
you make on Saturday or Sunday will be lost by Monday.
Other systems supported by the SIS team will be available as usual: SSOL,
SIS Desktop Reports, InfoMaker, and the Voice Response Unit.
We apologize for the inconvenience this may cause. Thank you for your
patience as we support AIS in implementing this much-needed system update.
Please let us know (at infoline@columbia.edu) if you have any questions.
March 10, 2003
Last week, the mainframe Student Information System experienced an
unprecedented incident in its decade-plus existence. Despite the best
efforts of both AIS and SIS staff, our users lost between one and four days
of system time when Tuesday's online work was lost, the system was
unavailable on Wednesday, and most or all of Thursday and Friday were
required for recovery and re-entry of Tuesday's work.
AIS staff are still investigating the exact circumstances and cause of the
problems, and we at SIS are working with them to define procedures to
prevent a recurrence. In the meantime, I have included below an outline of
the events of last week. I am also compiling a more detailed document with
examples of the data corruption, including more specific illustration of the
technical problems we faced in trying to save Tuesday's online work (which
led to our decision to abandon the effort as too risky). If you would like
to receive a copy of that document, please let me know and I will send it to
you as soon as it is complete.
Finally, I extend the apology of AIS and offer my own to all of you for the
extensive interruption of business you suffered last week. Our Admissions
and Financial Aid communities were especially hard hit, as March is their
peak processing month, and I know many others were very seriously
inconvenienced. At the same time I thank you all for your understanding,
and for the good will that so many of you sent our way while we worked to
recover the system as quickly as possible. We will work hard to re-build
that spirit of cooperation that it is our good fortune to share with our
colleagues, both users and partners alike.
Please do not hesitate to contact me via phone or e-mail if you would like
to discuss this matter further, or to offer any comments or suggestions for
the SIS team.
* * * * * * * * * * * * * * *
Summary of Events during the week of March 3, 2003
On the night of Monday, March 3rd, a series of errors during the processing
of batch jobs for the mainframe system left the database in a significantly
corrupted state, though the damage was not then evident to the AIS staff who
manage the nightly SIS production cycle. During the afternoon on Tuesday,
the SIS team began receiving reports of strange data discrepancies that our
mainframe users were discovering during the course of their daily online
work. We began investigating and called upon our AIS colleagues for
assistance; during the late afternoon and early evening hours the team
confirmed that the errors of the night before had in fact caused serious
damage.
AIS and SIS staff worked until shortly after midnight attempting to
determine the extent of the data corruption and a course of action to
correct it, but were unable to do so. At 12:06 AM on Wednesday, March 5th,
we made the decision to cancel the nightly production cycle; six hours later
we decided to keep the system unavailable for Wednesday online (i.e., user)
activity, at least until 2pm. Both actions were taken to prevent the
potential for further degradation of database integrity. At that time, our
hope and expectation was that further analysis would yield both a recovery
plan for the corruption that had been introduced on Monday night, and a
means by which we could preserve the online work our users had done all day
on Tuesday.
By mid-day Wednesday it was apparent that Tuesday's online work could not be
preserved by systematic means without the potential for introducing new data
discrepancies. The Director then called a halt to those efforts and
instructed the team to concentrate on recovering the database to its
last-known state of reliable data integrity. The decision was subsequently
affirmed by the Interim Deputy VP of AIS and the Executive Director of SIS.
At approximately 6:30 PM on Wednesday, March 5th, the Student Information
System was recovered (and made available), in the correct state it would
have shown had there been no errors on Monday night. The system came down
as usual at 10:00 PM, and the normal Wednesday night production cycle was
run. The SIS team worked closely with all affected users on Thursday and
Friday to identify lost work from Tuesday, and to facilitate its re-entry
into the SIS.
March 5, 2003
As of 6:30 p.m. Wednesday, March 5, Mainframe SIS is available for processing. It will be available until 10:00 p.m. this evening. User-Submitted Batch Jobs via SISUMENU are being accepted and will be processed tonight in the overnight production cycle.
SIS currently reflects correct information that includes all of Monday's manual (on-line) transactions and all Routinely Scheduled Monday Night Batch Processing. It does not include any information manually keyed into SIS during the day on Tuesday. If you need assistance identifying the manual entries your office may have made on Tuesday, please contact us at infoline@columbia.edu. We have methods of identifying your Tuesday transactions and will provide all assistance necessary until each and every SIS User is satisfied with the state of their data.
Monday night's user-submitted (SISUMENU) batch job requests are *not* reflected in SIS and will need to be re-submitted. In addition, any jobs submitted on Tuesday were canceled. SIS is in the process of contacting each user who submitted jobs via SISUMENU on Monday or Tuesday. If you submitted a job, you will be contacted shortly.
Finally, the Report Server, which supplies data for SIS Desktop Reports, ad-hoc SQL reporting, and Student Services On-Line (SSOL) will continue to be current and accurate as of Sunday, March 2. During the day on the 6th, SIS and AIS are ensuring that by Friday, March 7 the Report Server will be up-to-date and completely accurate.
If you have any questions at all, please do not hesitate to contact us at infoline@columbia.edu. Thank you so much for your patience and your support and cooperation throughout today.
October 01, 2002
I'm very pleased to announce that, effective immediately, availability of
the mainframe SIS system has been extended to 10pm Monday through Thursday.
(For now, the Friday and Saturday schedule remains unchanged.)
These additional 4 hours per week of online system time have been gained
thanks to recent software improvements made by our colleagues in AIS. The
new software installed by AIS has significantly shortened the time necessary
for overnight processing, specifically the time needed to perform backups of
the system. The time saved is then used to keep SIS up later in the
evening.
The complete SIS online schedule is as follows:
Monday-Thursday 7:30am-10:00pm
Friday-Saturday 7:30am-6:00pm
 |
