All posts by admin

UPS Failure , Several Servers Shutdown

The Matrix UPS providing power to the SOUTH RACK failed at about 8pm, Sun, 3/6. Discovered about 10:45pm. NetSYS tech support switched UPS to BYPASS mode about 11pm.

Servers known to be affected include: Calendar, Mail, DNS, CCTV, FS, FSAPPS. Exact status of servers unknown.

NetSYS techs discussed strategy and decided to take down all servers with single-source powersupplies and switch UPS back to OPERATING mode. This was done and UPS responded appropriately and all seemed to be working normally. Battery at 98%.

Special procedure done to the iSeries (AS400) box (a single-source power supply server). NetSYS worked with Loretta (via phone) to powerdown the unit. Pressed the right button on face plate and power-up proceeded to LOGIN screen on console within 5 minutes.

As a precaution DNS server moved to NORTH Matrix until we are sure all problems with SOUTH Matrix are resolved.

Some additional servers had to be restarted to insure they were running properly. In the course of doing these server restarts it was found that the MAIL server would not start properly. This may be due to changes made last week for the LDAP services. See separate Outage log setup for the MAIL server.

ST crashed

ST crashed today when doing some general maintenance.

When DSrepair finished, it showed on error corrected. Hit the ESC key and the notification went away, but the screen never came back to normal. CPU was at 90+, monitor showed nothing abnormal in busiest threads – a lot were 200,000 – 300,000. Had to power off and restart.

NS server unavailable

Could not log in to NS. The console reported that console.log could not be written. The server had not abended, and did not show abnormal stats in Monitor. A forced “down” got the server almost back to DOS, where Ctrl-Alt-Del reset it. On boot it stopped at the RAID controller BIOS, and the activity light for the lower left HD was on solid. After power-cycling, the server booted normally.