| Start Date | Start Time | End Date | End Time | Date Reported | System Affected | Impact to Users | Issue Description |
| 11/15/2009 | 8:00am | 11/16/2009 | | 11/15/2009 8:10:08 AM | Web (vcu.edu) | Users can't login | E-ID manager website: http://eidmanager.vcu.edu/ is down. When attempting to access site, the following error is displayed:
Internet Explorer cannot display the webpage
Hobbit showing RED for eID_Manager, eidmanager-lb, |
| 11/12/2009 | 11:15am | 11/12/2009 | 12:30pm | 11/12/2009 11:26:24 AM | Network | Other | 11:18am Alan Goodman of VCU Network Services called the NOC to inform us that the router closet at the Siegal Center is flooded and there is no ETA of when it will be fixed.
12:30pm Tesh Patel of VCU Network Services called NOC and stated that the Siegal Center router is fixed and is up and operational. |
| 11/01/2009 | 12:30pm | 11/01/2009 | 3:30pm | 11/1/2009 12:57:53 PM | Network | Cisco router down | Network Problem - GRANT-220-S1 switch at 2nd Floor, William H. Grant House, 1008 East Clay Street connection went down @ approximately 12:30PM. |
| 10/14/2009 | 4:00pm | 10/14/2009 | 7:00pm | 10/14/2009 5:35:23 PM | Network | Internet access slow | 4:00pm NOC noticed several red Hobbit Network alerts for Internet Links & INOVA. We proceeded to contact Vernon Williams the on-call VCUNET Network Services tech. 5:06pm Vernon called back with an update and mentioned the problem is our Verizon link (VORTEX) is down. He contacted VA TECH who then contacted Verizon. This is also affecting ODU. The is no ETA of when Verizon will fix link. However our MCI link is up but is only providing about one quarter of our total capacity, internet access will be slow.
______________________________________________________________________Per confirmation from Vernon Williams, the Network Connectivity to Level-3 and LambdaRail was resolved around 7pm. |
| 09/09/2009 | 9:30pm | 09/09/2009 | 10:45pm | 9/9/2009 10:19:13 PM | Network | No connectivity from a specific building | Network On-Call Engineer has confirmed from Virginia Power that there is a Power Outage in the following Areas - There is no Estimated Time of power reconnection per CE: Areas of concern are Monroe Park Campus & VCU Main Links: Break down is listed as follows:
The 1st Floor, Student Commons, 907 Floyd Ave; the Pollack-Basement-S2 & the Pollack-Basement-S3, both on 325 North Harrison Street, Scherer Hall; the Ackell Hall (1100wb-110-S1, 1100wb-110-S2, 1100wb-110-S3, and 1100wb-110-S4); The VCU Police Station (940-wgrace-119a-S2); All Broad and Belvidere Apartments, 700 West Broad Street; GSthea-121-S1; 1314-Wmain-H1-air-br350; 1312-wmain-S1; and 10NB-basement-S1.
*******************************************************************
10:52pm The power outage is resolved. All areas are reconnected. |
| 09/01/2009 | 5:00am | 09/01/2009 | 3:30pm | 9/1/2009 10:29:50 AM | Network | Other | 09/02/09 15:30 pm Adjustments were made on the router to alleviate the problem for now. The problem could reoccur if the router reboots. Network engineers know what to look for and are able to handle if it does happen. They are still looking to prevent further occurrences.
7:30am Network will know for sure if the tuning of router reconfiguration worked around 10am this morning when traffic on internet is very heavy
9/1/2009
12:46 pm - Cisco is on site and is working with Network Engineers and Cisco TAC in Raleigh on this problem. More later.
10 am . NOC received a called from Network Services stating that the issue that arose from Sunday's maintenance is ongoing. The primary router at 900 E Main is running at 100% CPU utilization. This is affecting all traffic (internet and servers) at 900 E Main. The maintenance this morning did not fully correct the problem. Network Services is still working with Cisco on the problem.
***************************************************************************************************
|
| 08/24/2009 | 4:00am | 08/24/2009 | 2:45pm | 8/24/2009 7:38:00 AM | Network | Other | Update- 2:45 p/m 8/24/2009 Everything is now back up and operational. Water had leaked from the street to conduit. All equipment in the basement of Seigel had to be replaced due to water damage.
We are currently experiencing several router and switch outages, located in and around the Siegel Center area. This includes the Siegel Center at 1200 W Broad St, Fine Arts Building at 1000 W Broad St, Ackell Hall at 1100 West Broad St and the Sports Medicine Building at 1300 West Broad St. Other locations may also experience problems connecting to the network, such as Harris Hall, Cabell Library, the Recsports Center, bow street parking deck and the laser connection at 612 Lombardy St. The problem is due to flooding in the Siegel Center, from last night's thunderstorm and torrential rain. Technicians are working to correct the problem, and we currently have no estimated time when services will be restored. We will update as information becomes available to us. |
| 08/14/2009 | 7:00pm | 08/25/2009 | 6:00am | 8/14/2009 7:36:39 PM | Network | No connectivity from a specific building | Per Tesh Patel (Network Engineer) concerning the Cabel Library Shutdown - Due to the installation of an Emergency Generator by the Facility Staff, there will be no Network Connectivity to this building, on 8/14/2009, beginning from - 7pm through 8/15/09 - 6am. |
| 08/06/2009 | 6:30pm | 08/06/2009 | 12:00am | 8/6/2009 6:56:53 PM | Other | Other | The Telecommunications Services Group has escalated the following issue to Verizon; External callers are unable to reach customers on the Monroe Park Campus. External callers have reported they are getting a fast busy tone or an "all circuits are busy" message when attempting to reach the Monroe Park Campus office. This trouble issue is being tracked under Pinnacle trouble ticket 25720-1.
_____________________________
Problem resolved at midnight, per Robin Roane. |
| 08/03/2009 | 1:15pm | 08/12/2009 | | 8/3/2009 1:46:03 PM | Other | Other | The Micros POS client loader service terminated unexpectedly. Hobbit displayed CPU at 100% and netvupoint service had stopped. 1:19 Went to inform sys admin and by the time I got back all services had stopped.
1:23 Syst ADmin currently accessing the problem. |
| 07/29/2009 | 1:15am | 07/30/2009 | 10:00pm | 7/29/2009 1:46:02 PM | TSM | Other | 1:15 pm. Received message Accessor A failed, cartridge could not be released from gripper on the library console. Library had gone into pause mode.
Opened the enclosure door and removed the stuck tape from gripper. Tried to clear the intervention but the library would not go into AUTO.
1:20 Spoke to zoltan and we agreed to place to service call to CSI. Reference# 255888.
1:28 Bill Pittman returned the service call and said he would contact the guys in DC.
05:10pm Bill Pittman and Dan Watta arrived to work on ATL. What they found was the track the robot works on was off alignment. They re-aligned track and had to re- teach the Atl. ATL is now in auto mode and functioning properly.
09:00pm Called was placed to Zee to inform him CSI was finished with ATL. Zee said to make sure the ATL is accepting all the tapes that were ejected out of machine earlier. ATL accepted all tapes and is running fine. Robot is running up and down track. Machine is working in auto mode.
10:00pm Bill and Dan exit Computer Center.
|
| 07/17/2009 | 11:30am | 07/29/2009 | | 7/17/2009 12:57:21 PM | Other | Other | 12:50 am. Tom Compton called to report a DHCP server failure. He stated it had been reported and a new unit is on order. The ETA is monday. We have redundancy so it should not be apparent to anyone. |
| 07/09/2009 | 7:45am | 07/09/2009 | 11:00am | 7/9/2009 8:21:14 AM | Network | Other | 7/9/2009 11:00 am The network is up and operational. The cause of the problem as reported by Network Services is the IPS box started denying certain traffic. It is not known at this time why the IPS did this but a bug in the code is suspected. The IPS has been removed and a TAC case with Cisco has been opened.
7/9/2009 07:45 am We are currently experiencing outages involving several servers and applications. The problem is being investigated and updates will be made as they become available.
|
| 05/27/2009 | 9:15am | 05/27/2009 | 4:15pm | 5/27/2009 1:53:17 PM | Network | Other | 4:15pm, NOC received an notification from VCUNetwork Services that the Cary street parking is up and operational.
Cary Street parking lost connectivity at 9:15 a.m. ETA 5/29 sometime late morning.
|
| 05/12/2009 | 4:45pm | 05/13/2009 | 6:30pm | 5/12/2009 7:50:48 PM | Network | Other | 05/12/09 @ 4:50pm – The Network Operations Center noticed connectivity to all phones and computer systems had been interrupted. Contact was immediately made with Network Services. A Network Engineer stated that the outage was caused by an issue with a main router. Network Services was immediately aware of the situation and was working to resolve the problem.
Several University systems were affected including Internet, e-mail, VCU portal, Banner, Blackboard and the LDAP authentication service. Internet, telephone and mail services were restored after 10 minutes. Access to the Banner system was not restored until 7:30pm. Banner appears to have returned to service and seem to be in proper working order at this time. |
| 04/27/2009 | 11:15am | 05/04/2009 | | 4/27/2009 11:54:43 AM | Other | Other | 4/28/2009
UCC Technical support restored the data from survey.vcu.edu to Inquisite2. The certificate from the old surveey hardwaree would not function properly on the new survey hardware so a new certificate was generated by the NOC and installed by Terry Leonard. Server is up and operational. 2:25 pm
4/27/2009
correction:
Informed by UCC technical support that Survey.VCU.EDU has suffered a catastrophic hardware failure. Data is being restored to a new server. Technology Services staff are working to correct this problem as quickly as possible. Survey.VCU.EDU will be unavailable until the new production server has been configured. |
| 03/27/2009 | 1:45pm | 03/27/2009 | 3:15pm | 3/27/2009 1:55:56 PM | Network | Other | Hobbit indicated a loss of connectivity for Cabell 4th floor.Contacted network services oncall. Was told that one of the staff member were already working on the problem. No ETA given.
2:15 per Vernon Williams. power knocked out by electricians. should be restored within the hour. 3:15 p.m. Connectivity has been restored to Rooms 421 (S4 and S-5) Cabell. No further problems. |
| 03/25/2009 | 7:30pm | 03/25/2009 | 8:30pm | 3/25/2009 8:17:26 PM | Other | Other | 0730pm Noticed ASA in hobbit red, drilled down on ASA found poca-asa.5540-webvpn(WEBVPN) is down due to no connectivity.
07:38pm Mike O'Neil called said he is aware of WEBVPN being down and he is on his way in to resolve the problem. Called help desk to alert them of problem and let them know Mike is on his way in to fix.
Tried to log on to VPN and received message 'connection attempt has failed due to unresolvable host entry'. Mike also instructed me not to alert Vernon because he is coming in to resolve.
08:10pm Mike O'Neil have arrived to resolve problem. No further problems. |
| 03/11/2009 | 7:15pm | 03/11/2009 | 12:45pm | 3/11/2009 12:04:53 AM | Mail | Other | 3/11 12:40pm spoke to Ken Perdue no further issues. The 5 users affected have been taken care of.
Around 7:15pm, users began calling in with concerns about receiving an email: 'Your Account May Be Deleted In Two Days Time (the entire email is referenced on the VCU Alert Page)'.
I responded by first opening a Remedy Ticket (HD182002) addressed to Lotus Group, then calling Margie Rigsbee and Ken Perdue to research the problem.
_______________________________________________________
Per Connie Lowery and Ken Perdue: The five users that responded to the scam email has been deactivated from accessing their email until further research is done. |
| 03/06/2009 | 2:45pm | 03/09/2009 | 2:30pm | 3/6/2009 3:03:51 PM | Network | Other |
3/9/09 resolved
3/6/09 @ 2:45pm - A power loss has occurred at 601 West Main Street, School of Engineering West Hall. This affects internet and phone connections. VCU Facilities, VCU Network Services and VCU Network Operations Center are aware. This power loss may affect other VCU facilities. We will update with more information as it becomes available. |
| 03/01/2009 | 7:45am | 03/01/2009 | 4:30pm | 3/1/2009 8:23:47 AM | SQL Server | Other | Resolved
Bill Rohr called the NOC to report that he could not access Kilkenny and none of his backups have run. He requested staff to contact the APPS team member responsible for the server.
NOC staff called Rick Roughton and left a message he has not responded. We received a call back from Bill Rohr who advised us to wait an hour and if Rick doesn't call, then contact the next DB tech on the list.
The database instance is ods1. According to Bill Rohr it appears the db never came back up after it's offline backup last night.
Contact was made with Rick Roughton at 9:15 am. He stated he just called Bill to notify him that he is now working to resolve the problem. Rick also stated that he is re-running the backups from last night. |
| 02/27/2009 | 6:00am | 03/03/2009 | 12:00pm | 2/27/2009 9:53:41 AM | LDAP | Other | 2/27 @ 9am - This morning after the scheduled Change Management maintenance ‘eDirectory Load balancer move to ACE blade’, some applications are not able to authenticate due to the load balancer change. The Network Operations Center is working with Network Services to resolve these issues.
Please contact the Network Operations Center (828-1802 or uccnoc@vcu.edu) with the name of your application and IP address of the server if you are experiencing related issues.
|
| 02/26/2009 | 7:00am | 02/26/2009 | 11:30am | 2/26/2009 7:15:56 AM | Other | Other | 2/26 @ 1pm - Network Services has resolved the issues with POCA-PE that surfaced during the replacement of the supervisor module. Network Services will be monitoring this situation.
2/26 @ 7:45am - Maintenance to replace the supervisor blade in the POCA-PE router this morning has affected the ACE blade load balancer. Network Services is working on the problem. |
| 02/26/2009 | 7:00am | 02/26/2009 | 8:30am | 2/26/2009 7:21:40 AM | Other | Other | Voicemail currently cannot be accessed by users. The Computer Center is working to solve the problem and will post updates as they become available. UPDATE: Problem resolved and voicemail is now available again. |
| 02/20/2009 | 11:00am | 02/20/2009 | 2:00pm | 2/20/2009 12:13:58 PM | Other | Other | 2/20/09 @ 3pm - UCC room temperature has been brought back to normal. We do not anticipate any problems over the weekend.
2/20/09 @ 12pm - There is a temperature situation at 900 East Main (Pocahontas Building). UCC Staff are working with DGS (Dept. of General Services). A partial solution is in place, but there is still a heat situation ongoing.
|
| 02/14/2009 | 12:00am | 02/18/2009 | 1:00pm | 2/14/2009 12:55:39 AM | Blackboard | Cannot log in | Due to unforeseen difficulties, Blackboard will not be back online for an extended time period. Technicians are currently working on the problem. We apologize for any inconvenience this may cause. Updates will be posted to the Blackboard system announcement blog at http://blog.vcu.edu/blackboard/, and on the IT Professionals Intranet, as they become available.
10:15 am Kenneth Purdue visited the NOC to report that there is a major failure with the Blackboard system. The server is down and will require the database to be moved to a failover system. Access to Blackboard will be available, but, will be noticeably slower than usual; due to the difference in performance in the regular production server versus the standby failover server. The data move will take place either on Sunday or Monday. We will be updated during the process. The Blackboard team has made notifications on the Blackboard site and through their normal notification process.
1:00 pm - Rick Roughton notified NOC staff in person that Blackboard should now be available to all users. Users may experience diminished performance due to the system now being located on a box with lesser performance capability. |
| 02/06/2009 | 11:15am | 02/06/2009 | 12:45pm | 2/6/2009 11:36:49 AM | Network | No connectivity from a specific building | 2/6/09 @ 12:45 The NOC received a call from Tesh Patel of VCU Network Services, stating that the WISM7 wireless device is online and operational. The device's npu/cpu has froze up and wasn't allowing packets to pass through. The impact to the users was at least 5 minutes, per Tesh .
--------------------------------------------------------------------------------------------
2/6/09 @ 11:15 The NOC noticed that the Wireless device (WISM7) displayed no connection in Hobbit. Tesh Patel of VCU Network Services called and stated it appears that the control board in this device is hung and he is working on this. This currently is affecting the wireless connectivity for the following locations: Ad Center, School of Business, Engineering (East & West), McGuire, Life Sciences, Sanger Hall. per Tesh there may be other buildings as well.
|
| 02/05/2009 | 8:00am | 02/05/2009 | | 2/5/2009 9:09:15 AM | Network | Other | 2-5-09 @ 9:05am - At 8:05am Cabell B-44 lost power, this has been reported to VCU Facilities and they are aware. The power outage affects 4 switches, so there will be no network connectivity in the areas supplied by these switches. Network Services is aware. |
| 02/01/2009 | 9:15am | 02/01/2009 | | 2/1/2009 9:19:52 AM | Network | CoVaNet down | Tom Compton called to report on an update regarding Level-3-ISP (Mission Critical), Abilene-10G-vlan30 (Non Mission Critical) and NLR-layer-3 (also Non Mission Critical) connections. There was a timeframe of downtime with Verizon to perform work but that time has elapsed at 9AM. Compton is waiting for an update from the VT NOC but still has not heard back from them. Compton will follow up with update ASAP. |
| 01/21/2009 | 1:00pm | 01/21/2009 | | 1/21/2009 1:18:33 PM | Blackboard | Other | 1/21/09 @ 1:04pm - NOC and TS Learning Systems noticed BBProd (Blackboard Production) http internal server errors, upon investigation; it was discovered that BBProdDB (BB62) crashed due to I/O errors on the /u01/ file system. The database needs to stay down while the file system is unmounted and FSCK is run. Sean will bring BB62 back up ASAP. ETA is 20 minutes. Call Sean for specifics @ 828-8415 |
| 01/13/2009 | 6:15pm | 01/15/2009 | | 1/13/2009 7:31:11 PM | LDAP | Other | Around 6:15, I noticed that hobbit was displaying that EDIR1 was having a service unavailable problem. After waiting about 15 minutes, with the server still in the unavailable status (LDAP not ok), I researched the Server Inventory sheet for EDIR1, and it listed Zoltan and Kay as the application people. Since I could not reach Z at the moment, I called Tracey, and she mentioned that Craig was the responsible person for the EDIR1 server.
_______________________
Upon contacting Craig, he mentioned that EDIR1 is a test server, and he was researching the problem.
___________________________________________
The Server Inventory sheet for EDIR1 may need to be updated.
|
| 01/09/2009 | 4:30pm | 01/09/2009 | 5:15pm | 1/9/2009 5:18:48 PM | Network | Other | 4:50p Vernon Williams (Network Services) called said Harris Hall router have a module failure and they are working on resolving problem. Several locations are affected by this outage. Harris Hall, Oliver Hall, Physical Sciences, Life Sciences, Mary and Frances Youth Center and Sch of Engineering are all affected; there maybe more locations affected South of Main St. and West of Belvidere when Network Services resets the router. Will update when more information is received.
5:17pm Vernon Williams called said he swapped board in router and all connectivity is restored. |
| 01/02/2009 | 9:30am | 01/05/2009 | 11:30pm | 1/2/2009 9:52:58 AM | Other | Other | Help Desk reports that their phone system is down and all calls are currently getting forwarded to the UCC. A request for service to the on-call network personnel has been made. No ETA.
No calls into UCC after 11:30AM. Issue has been resolved |
| 12/09/2008 | 11:30am | 12/10/2008 | 6:45am | 12/9/2008 12:10:04 PM | Network | Active Directory attacks | 12/10/08 - Work and testing completed on WiSM4 by Hitesh Patel. Problem resolved.
12/09/08 @ 11:42am - Network Services has reported that there is an authentication redirection issue with WiSM 4. Clients attempting to connect to access points under WiSM 4 are not being redirected to the WiSM authentication page.
This issue is not campus wide; it is affecting wireless connections in the following locations:
Cabell Library,
Gladding Residence Center,
Ackell Hall,
Hibbs Building
Network Services is aware and in contact with Cisco and is actively working on resolving this issue.
Long term resolution: Change management will be made for 12/10/08 @ 6am WiSM 4 - reset with a possible firmware upgrade. There will be no connectivity during this time. There is no ETA on how long the reset will take.
Short term work-around: clients wishing to connect and are not automatically redirected to the WiSM authentication page can manually connect to the authentication page:
https://wirelessauth.cns.vcu.edu/login.html
|
| 12/08/2008 | 2:30pm | 12/08/2008 | | 12/8/2008 2:51:16 PM | Network | Other | 12/08/08 @ 2:30pm - There has been a loss of power at the Monroe Park Campus Fine Arts Building (1000 West Broad Street). VA Power and VCU Facilities are aware and are actively working on this issue. This affects wired and wireless connections in this building. There is no ETA at this time of when the power will be restored. |
| 12/08/2008 | 2:30pm | 12/08/2008 | | 12/8/2008 4:00:17 PM | Network | Other |
12/08/08 @ 2:30pm - There has been a loss of power at the Monroe Park Campus Fine Arts Building (1000 West Broad Street). VA Power and VCU Facilities are aware and are actively working on this issue. This affects wired and wireless connections in this building. There is no ETA at this time of when the power will be restored. |
| 11/24/2008 | 10:30am | 11/24/2008 | | 11/24/2008 10:42:53 AM | Network | Other | 11/24/08 @ 12:00pm - Network Services has resolved the previously mentioned second issue with their 11/24/2008 10:45 AM - 11 AM Change Management. By adding a route statement for the Internet VRF to allow new Wireless subnet access to internal systems, wireless users can now access the internet. Network Services is still working to resolve the previously stated first issue, where users are having trouble authenticating to the wireless network by troubleshooting the Cisco Wireless Services Module (WiSM)
10:30am Per Network Services VCU is currently experiencing wireless access issues and are working on them.
The issues are:
1) Authentication - some users are not able to log on.
2) Uses who have authenticated may have problems getting to outside websites. |
| 11/24/2008 | 11:00am | 11/24/2008 | 2:45pm | 11/24/2008 2:55:40 PM | Network | Other | 11/24/08 @2:45pm Network Services has resolved the the wireless issues, per Tesh Patel.
11/24/08 @ 12:00pm - Network Services has resolved the previously mentioned second issue with their 11/24/2008 10:45 AM - 11 AM Change Management. By adding a route statement for the Internet VRF to allow new Wireless subnet access to internal systems, wireless users can now access the internet. Network Services is still working to resolve the previously stated first issue, where users are having trouble authenticating to the wireless network by troubleshooting the Cisco Wireless Services Module (WiSM)
10:30am Per Network Services VCU is currently experiencing wireless access issues and are working on them.
|
| 11/19/2008 | 3:45pm | 11/20/2008 | | 11/19/2008 4:23:19 PM | Other | Other | 11/19/08 @ 4:15pm - The DNS (Domain Name System) for Active Directory has started a rebuild process due to subsequent commands from a Windows Server Administrator. This may impact users' ability to authenticate / login to the domain. This is being monitored by the UCC Windows Administrators and is expected to be restored within the hour. |
| 11/17/2008 | 9:15am | 11/17/2008 | | 11/17/2008 9:17:26 AM | Network | Active Directory attacks | 11/17/08 @ 9:15am - Network Services has reported that the Harris Hall Wireless Service Module (WiSM) has lost connection. This WiSM controls 158 wireless access points in various dormitories and a few floors. There is no ETA at this time. |
| 11/10/2008 | 8:00am | 11/13/2008 | | 11/10/2008 8:57:45 AM | Other | Other | 11/12/08 @1405 - Cisco technicians have tentatively identified the IPS/IDS (Intruder Prevention/Intruder Detection) network device as the probable cause of the network issues causing network connectivity problems, including the Celerra (NAS, Network Attached Storage). We are now attempting to determine the best way to remove the impediment. Sean O'Keefe
11/12/08 @0750 - The Rose share, and other servers, are unavailable. The UCC and Network Services are working nonstop on these issues. Sean O'Keefe
11/12/08 @0610 - Router maintenance was applied @0400 this morning by Network Services to the Poca router located at the UCC. I am able to map to shared drives on the Celerra (Rose) at this time. Sean O'Keefe
11/11/08 @ 4:45pm - At this time, the problem is still unknown. There is a problem in a main router at the University Computer Center and maintenance will be applied at 4:00am Wednesday 11/12/08. For more information, see the VCU Planned System Changes (http://www.pubapps.vcu.edu/changes/plannedChange.aspx).
11/10/08 @ 3:30pm - Connectivity to mapped drives is intermittent, (short periods of successful connectivity, then connection failure). UCC staff is capturing activity to share with EMC (SAN vendor) and Network Services.
11/11/08 @ 10:50am - The issue has been escalated with EMC (SAN Vendor), there is still no ETA regarding a resolution. Please check the IT Professionals Intranet (https://www.ucc.vcu.edu/intranet/ITPros.htm) periodically for the current status.
11/10/08 @12:00pm - Windows Server Administration has contacted EMC (SAN vendor) and Network Services regarding this issue.
11/10/08 @8:00am - Users on both Monroe Park and Medical Campuses are reporting problems connecting to network drives. Windows Server Administration, Storage Area Network Administration and Network Services are aware and working to resolve this issue. |
| 11/07/2008 | 4:30pm | 11/10/2008 | | 11/7/2008 2:12:34 PM | Network | Other | 11/7 @ 2pm - Network Services installed a firmware upgrade on the Harris PE Router and has restarted the device. At the present time, there are no problems but Network Services will be monitoring its status.
_______________________________________________________
11/7 @ 10am - Intermittent VoIP phone problems persist. Network Services is working with Cisco to resolve this issue.
_______________________________________________________
11/7 @ 1am - Alan Goodman called and the problem has been resolved. Network has been stable for 1 hr and 45 min. There was an issue with the code to the VCU router.
_______________________________________________________
11/6 @ 9:30pm - Per Vernon, as of 9:30pm - Still trying to resolve the problem
_______________________________________________________
11/6 @ 6:50pm - Per Vernon, as of 6:50pm - Monroe Park was still having multiple failures. At this very moment, the router is restarting.
_______________________________________________________
11/6 @ 5:45pm - Tesh called around 5:45pm - Mentioned that the Internet connection was back up, but The Telecommunication crew were still checking on the IP Phones because its connection was not steady (going up and down).
______________________________________________________
11/6 @ 5:10pm - Vernon (Network Services) called - Report that the Harris Edge Router has been restarted
_______________________________________________________
11/6 @ 4:30pm - Network Problem - Hobbit reports connection loss at Seigel Center(4:28pm) and Harris Hall (4:33pm) - VCU Backbone reporting connection loss @ Seigel Edge.
Affecting - Cabell Core, Harris Core, Life Sciences, Seigel Center, Harris Hall, Harris Rooftop MAN: 1314 W. Main, 10 N. Brunswick, 8th N Harrision, 612 N Lombardy, Youth Center, Green Alley.
This may be a power outage, Network Services is looking into this and will inform
the NOC.
______________________________________________________ |
| 11/06/2008 | 4:30pm | 11/06/2008 | 1:00am | 11/6/2008 4:49:28 PM | Network | No connectivity from a specific building | 1am - Alan Goodman called and the problem has been resolved. Network has been stable for 1 hr and 45 min. There was an issue with the code to the VCU router.
11/6 4:30pm - Network Problem - Hobbit reports connection loss at Seigel Center(4:28pm) and Harris Hall (4:33pm) - VCU Backbone reporting connection loss @ Seigel Edge.
Affecting - Cabell Core, Harris Core, Life Sciences, Seigel Center, Harris Hall, Harris Rooftop MAN: 1314 W. Main, 10 N. Brunswick, 8th N Harrision, 612 N Lombardy, Youth Center, Green Alley.
______________________________________________________________________
This may be a power outage, Network Services is looking into this and will inform
the NOC.
Vernon (Network Services) called around 5:10pm - Report that the Harris Edge Router has been restarted
_____________________________________________________________________
Tesh called around 5:45pm - Mentioned that the Internet connection was back up, but The Telecommunication crew were still checking on the IP Phones because its connection was not steady (going up and down).
______________________________________________________________________
Per Vernon, as of 6:50pm - Monroe Park was still have multiple failures. At this very moment, the router is restarting.
______________________________________________________________________
Per Vernon, as of 9:30pm - Still trying to resolve the problem |
| 11/04/2008 | 2:30pm | 11/05/2008 | 3:45pm | 11/4/2008 3:56:57 PM | Banner | Other | 2:30pm The Network Operations Center noticed a rapid increase in computer room temperature in a short period of time. The Department of General Services (DGS) was notified and axillary cooling mechanisms were put in place. DGS has addressed this issue and the increase of temperature has ceased.
Systems in racks UCC65-UCC67 were effected due too breakers tripping caused by excessive heat load. We have restarted all servers in those racks and are asking users to verify functionality of that equipment. |
| 11/03/2008 | 8:45am | 11/03/2008 | 9:00am | 11/3/2008 10:51:42 AM | Network | Other | 11/3/08 - At 8:45am, Network Services replaced a Cisco switch in the switch stack due to over-heating, which switches for multiple network services and was believed to be hot-swappable. It turned-out to be non-hot-swappable, so additional configuration was needed to bring it into production. There was a brief time period that the switch stack was not working, resulting in loss of connection for multiple systems. Per Tom Compton, this has been resolved. |
| 10/31/2008 | 8:45am | 10/31/2008 | | 10/31/2008 9:16:47 AM | Network | Active Directory attacks | 12:30 p.m. notified by network services that power had been restored seems like to all buildings except McAdams House 914 West Franklin Street.
9:50 Notified by network services that the outage is due to vehicle accident that has taken out a sub-station.
At 8:45am a power outage was reported at the following locations:
Technology Administration Building - 701 West Broad St.
Department of Psychology Center for Psychological Services - 612 North Lombardy St.
Ramz dormatory - 933 West Broad St.
Student Media - 812 West Broad St.
Treasury Services - 807 - 809 West Broad St.
No ETA on power restoration - this affects network and phone connections. |
| 10/31/2008 | 11:45am | 10/31/2008 | | 10/31/2008 12:03:12 PM | Network | Other | 1:28pm all power has been restored
12:00 - Per Network Services - Outage affecting the following area:
South side of Grace Street to Cary Street and West of Belvidere
12:00 p.m. power has been restored to all areas except McAdams House
11:45am - Other areas now reporting power outage or network connectivity issues due to power outage:
Mary and Francis Youth Center - Cary Street Field
Green Alley
Ginter House
McAdams House
Life Sciences
Cabell Library
Shafer Street Playhouse
Technology Administration Building
Blanton House
Hunton House
Williams House
Millhiser House
Stagg House
Valentine House
Alumni House
Thurson House
Bird House
White House
Buford House
Stokes House
Harris Hall
Oliver Hall
School of Engineering - West
11:30am - Per Keith Deane - Power restored. All systems reporting up.
9:50 Notified by network services that the outage is due to vehicle accident that has taken out a sub-station.
At 8:45am a power outage was reported at the following locations:
Technology Administration Building - 701 West Broad St.
Department of Psychology Center for Psychological Services - 612 North Lombardy St.
Ramz dormatory - 933 West Broad St.
Student Media - 812 West Broad St.
Treasury Services - 807 - 809 West Broad St.
No ETA on power restoration - this affects network and phone connections.
Start: 8:45am
End: 11:30am |
| 10/23/2008 | 11:30am | 10/31/2008 | | 10/23/2008 11:54:37 AM | Banner | Other | 11:26am 10/23 UCC NOC noticed a Hobbit alert for the Banner Production system. They contacted the UCC DBA/Banner support staff, who are looking into this. Per the DBA staff, the entire Banner Production system is currently down due to a hardware communications failure. The initial estimate is that the system will not be available until 2:30 pm this afternoon.
12:30pm The Banner production is up and operational
|
| 10/16/2008 | 6:45am | 10/17/2008 | 7:00am | 10/16/2008 6:54:16 AM | Banner | Other | We are having problems with CNAC Manager (main server Clean Access manager) which will affect user ability to connect to/through Clean Access. Tom Compton is looking into the problem and we will provide updates as they become available.
7:00 am - Problem identified as a firewall issue and resolved by Tom Compton. |
| 10/10/2008 | 10:00pm | 10/13/2008 | | 10/10/2008 7:47:15 PM | Network | Other | Update 10/11/08
Tesh Patel called to inform UCC of latest update. Wireless access has been moved to another unit to allow for internet access. DHCP issue. CISCO is aware of the problem. No ETA.
______________________________________________________________________
7:30pm Update:
Wired Access:/
The Wired access problem in the Cary and Belvidere dorm has been cause by the failure of the Clean Access Server for that area. Wired service to Cary and Belvidere Dorm has been restored by bypassing the Clean Access Server. Students can now directly connect to the Network from the Cary and Belvedere Dorm without a sign on.
______________________________________________________________________
Wireless Access:/
For the Wireless problem, we have already bypassed the failed Clean Access Server. Wireless access has mostly been working, but one controller as been failing. We are currently moving the Critical Access Points particularly those in the Dormitory, from the failing controller to working Controllers. This should recover most of Wireless access.
All work should be completed this weekend.
|
| 10/10/2008 | 10:00pm | 10/13/2008 | | 10/10/2008 3:18:36 PM | Network | Other | The Cisco Clean Access Manager system failed Wednesday night. As a work-around, Network Services removed Clean Access authentication (for wireless only) and put WiSM eID authentication in place. Network Services is looking into this. |
| 10/08/2008 | 9:30pm | 10/10/2008 | 1:30am | 10/8/2008 11:15:25 PM | Network | Other | Failure to connect to cisco clean access for wireless. Entire wireless system using clean access is down, no activity coming across line. Vernon Williams is aware and is working to resolve problem. Problem was resolved a little before 1:30 am. |
| 09/24/2008 | 2:00pm | 10/01/2008 | | 9/24/2008 2:35:32 PM | Network | Internet access slow | 10/01/2008 11:15am The user has been contacted and computer has been examined by VCU IT security. User has been granted network access again.
9/26/08 - 12:30 VCU Network Services & VCU IT Security have
identified the user. The user's network access has been cut. VCU Network Services and the VCU SNOC are monitoring this situation.
9/26/08 - 10:30am - The Secondary DNS servers are experiencing similar high CPU utilization which can affect a user's ability to resolve outside addresses, they are now operational. VCU Network Services and the VCU SNOC are monitoring this situation.
9/25/08 - 2:00pm - The Secondary DNS servers are experiencing similar high CPU utilization which is affecting user's ability to resolve outside addresses. VCU Network Services and the VCU SNOC are looking into this.
9/25/08 1:00pm The Secondary DNS servers are up and operational since 5:15pm yesterday. VCU Network Services will be monitoring the servers throughout the day.
9/24/08 2pm The Secondary DNS servers with IP's of 128.172.1.2 & 128.172.90.10 have high CPU utilization which is affecting VCU's ability to resolve outside addresses. VCU Network Services is looking into this |
| 09/17/2008 | 11:00am | 09/23/2008 | | 9/17/2008 11:21:07 AM | LDAP | Cannot log in | 9/23/08 - eDirectory All production servers (7-14) have the hot fix applied. At this point we are in a wait and see mode per Vendor. Also two of the load balanced servers have been up since Friday.
So far edir8 has been up since 11:30am and edir10 since 1:30pm. The others have been running since 4:00pm.
I will keep you updated if anything changes.
At this there is no ETA on resolution.
9/17/08 - eDirectory is experiencing intermittent crashes due to unknown causes. UCC staff is in contact with the vendor regarding this issue. This may affect application authentication at times. Users experiencing eID authentication issues should reattempt after 10 minutes.
At this there is no ETA on resolution.
|
| 09/16/2008 | 3:15pm | 09/17/2008 | | 9/16/2008 3:42:52 PM | Network | No connectivity from a specific building | FMD Operations reports a power outage on West Broad and Harrison Streets. No ETA of power restoration. This affects the Seigel and Lafayette routers. VCU Network Services is aware.
This affects the following buildings:
VCU Police (940 West Grace Street)
Ackell Hall (1100 West Broad Street)
VCU Dance Center (10 North Brunswick Street)
Broad and Belvidere Apartments (700 W. Broad Street)
Broad and Belvidere Apartments, 700 W. Broad St.
1312 W. Main Street
1314 W. Main Street |
| 09/13/2008 | 12:45pm | 09/15/2008 | | 9/13/2008 1:47:53 PM | Netwo |