IT Organization and Data Center Support
Part Two - Challenge #12

Background:

Large Enterprise IT organizations are divided into specific roles with distinctive responsibilities.

  • Application Development
    * Maintain existing programs and write new programs in support of the business
    * Supported by all roles that follow
  • Data & Disk Storage Administration
    * Maintain and monitor disk storage space used by application development and production processing
    * Define logic and constructs to automatically assign disk storage for new data sets based upon requesting application or user
    * Backup important data and be prepared to restore data upon request
  • Data Security Administration
    * Secure and monitor protected data sources
    * User ID management and authorizations
    * Secure data tranmitted over networks
    * Selectively encrypt data
  • Data Base & Transaction Services Administration
    * Support and services related to complex Data Base & Transaction Services technology
  • Systems Programming
    * Install and maintain hardware, operating system software and other vendor software
    * Responsible for problem identification and resolution of operating system failures operational issues
    * Implement and support procedures used by all IT organization roles
    * Responsible for system performance and system capacity
Far from least of these roles and responsibilities is

  • Operations - Systems and Network
    * 24x7 Data Center control, monitoring, and problem resolution related to hardware, operating system, and network
    * Responsible to all users for continued IT service
    * Most operational procedures and processes are created by System Programmers
    * Receives support from any of the above distinctive roles depending upon the problem
  • Production Analysts
    * 24x7 Data Center control, monitoring, and problem resolution related to application program processing
    * Maintain production schedules involved many thousands of jobs that run daily
    * Support and services related to automation of operation and production processing
    * Receives support from any of the above distinctive roles

An interesting aspect about these technology roles and responsibilites is that the skills can be applied to a wide variety of industries.

The previous challenges are core z/OS skills that prepared you for challenges related to these specific roles. The best place to start a journey of mastering a large enterprise IT organization is the operations and production analyst area. This is where everything comes together and has the broadest view of the technology, roles, and responsibilities.

Challenge:

Successful completion of this challenge has data in hlq.P2.OUTPUT(#12).

z/OS operation staff must be familiar with operating system commands for immediate problem analysis related to reported problems.

Scenario:

  • Large Enterprise Executive Assistant (EA) is very aware that an unscheduled mainframe outage costs the company a small fortune every minute that the processing environment is unavailable to the general public.
  • EA receives a message on the screen while interacting with the mainframe that reads, "Catastrophic System Failure".
  • EA is alarmed that this message could represent financial harm to the organization.
  • EA calls the production operation help desk, the nerve center of critical processing activity, and reports the "Catastrophic System Failure" message.
  • Help desk immediately informs the lead operator that EA is reporting a "Catastrophic System Failure".
  • The lead operator now owns this reported problem.
  • None of the automated processes or automated operational alert mechanisms is reporting a system problem.
  • If the lead operator is unable to isolate and resolve a problem, the procedure is for operator to contact systems programming.
  • The lead operator is expected to manually check on the z/OS environment with a series of actions prior to contacting the systems programming team for assistance with the reported problem.

Output from the commands below will contain answers to questions that the systems programmer will ask you, the lead operator.

Use the SDSF console facility, then open a user log to conveniently look back through command output previously entered.

=sd ; ulog

Navigating the system log uses syntax that is available when reading a data set using ISPF such as

  • find
  • find prev
  • up (F7)
  • down (F8)
  • left (F10)
  • right (F11)
  • top (m F7)
  • bottom (m F8)
  • CSR for Scroll amount

Enter the following commands as if you are the lead operator just looking around the z/OS environment for anything abnormal. Don't worry if you enter a command incorrectly. It will not affect the ability to complete the challenge 100% correctly. Just enter the proper command again.

  • /d t
  • /d r,l
  • /d grs,all
  • /d m=cpu
  • /d m=stor
  • /d m=chp
  • /d asm
  • /devserv qdasd,type=all
  • /d alloc,grplocks
  • /d omvs,limits
  • /d omvs,file,exception
  • /d omvs,waiters
  • /$jddetails
  • /d autor
  • /d ios,stop
  • /d a,l
  • /d msgfld
  • /d prog,exit
  • /d prog,apf
  • /d prog,defaults
  • /d ssi,status=active
  • /d ppt,all
  • /d net,id=osatrl1e,e
  • /d tcpip,,n,route
  • /d tcpip,,n,devlinks

Entering ulog again will show all command output in our personal user log.

z/OS Display commands and descriptions
z/OS DEVSERV command and description

Enter the following to immediately engage with the on-call systems programmer which simulates a cell phone text message facility:

tso zhelp
Tip: There are just 3 questions. The first time, just record the question and press enter to read and record the next question.

Read each question and enter to proceed to next question. The output from the commands previously entered include the answers to the questions. Review the ULOG or SYSLOG to find the answer to each question and enter tso zhelp providing answers. You may enter tso zhelp as many times as needed. Each new execution of zhelp replaces previous zhelp answers.

Answers are written to hlq.P2.OUTPUT(#12). View hlq.P2.OUTPUT(#12) to validate your answers

Attention MAC workstation contestants:
If hlq.P2.OUTPUT(#12) lines appear blank, then enter right of F11 and scan for numbers.
If the 3 numbers are on the far right side of the line, all is ok, F3. Score routine will find the numbers.
If the lines are totally blank, then enter answer to question #1 on line 1, answer to question #2 on line, and answer to question #3 on line 3 .

Next: Challenge #13