| |
|
Date
&
Initial |
W/P
Ref |
Additional
Comment |
| |
Review
of System Development Approach |
|
|
|
| 1. |
Obtain
a copy of the system development methodology (SDM)
standards/policies for the application in development. Evaluate
the reasonableness and completeness of the methodology. A
good SDM should have, at a minimum, the following sections:
- cost
- benefit analysis
- risk
analysis and evaluation
- user
requirements documentation
- system
specifications
- conversion
plan & results
- testing
plan & results
- rollback
plan
- approval
documentation
- schedule
and budget management
|
- |
- |
- |
| 2. |
Compare
the system methodology employed to the basic steps listed above
and to the University standard. Evaluate the documentation for
completeness and reasonableness. |
|
|
|
| 3. |
Obtain
a copy of the option analysis that was used to select the
solution being implemented. Evaluate if it clearly defined/documented:
primary
requirements for the new solution (including minimum functionality,
processing requirements, implementation schedule, and cost),
- outline
of options,
- solution
selection criteria,
- risks
and benefits of options considered,
- estimated
cost of options at a high level and in greater detail for
prime options,
- who
is the project sponsor,
- who
is the system owner, and
- who
needed to approve the selection decision.
|
|
|
|
| 4. |
Obtain
a copy of documentation demonstrating the project was approved.
Evaluate if it clearly defined/document:
- who
approved it,
- the
project budget,
- the
project schedule,
- deliverables
expected from the system (i.e., a high level description
of what functionality the system will provide,
- roles
and responsibilities for managing the project, and
- how
and when variance from expectations must be reported to
top management for strategic decisions.
Evaluate
if the project approver had appropriate authority to approve
the project.
|
|
|
|
| 5. |
Obtain
project budget to actual expense reports. Evaluate if budget
amounts reconcile to budgets originally approved. Evaluate if
process effectively captures all expenses associated with the
project and accurately projects future expenses. Evaluate if
significant negative budget variances are reported to top management
in a timely manner so they can make effective strategic decisions.
Evaluate the reasonableness of action being taken to manage
budget variance. |
|
|
|
| 6. |
Obtain
a copy of the project schedule. Evaluate if it serves as an
effective tool to assure target dates and variances from plan
are quickly identified and effectively managed. Evaluate if
significant negative schedule variances are reported to top
management in a timely manner so they can make effective strategic
decisions. Evaluate the reasonableness of action being taken
to manage schedule variance. |
|
|
|
| 7. |
Obtain
a list of expected deliverables from the system and information
on the process to prioritize. Evaluate the process used to prioritize
those deliverables and schedule their implementation. Evaluate
if the list appears complete and reasonably prioritized. |
|
|
|
| 8. |
Obtain
information on the process used to assure control and integrity
issues are being effectively addressed in the new system.
Evaluate if the process mapping business control issues to
the business processes is supported by the system requirements
list.
- Specifically
evaluate procedures to assure:
- reasonable
balancing and reconciliation procedures will be addressed,
- critical
edits and exception reporting is put in place,
- necessary
transaction audit trails exist, and
- capabilities
to limit access by function to assure proper segregation
of duties.
Obtain
information on how the project manager is assuring that effective
control of the business process improves with the new system.
|
|
|
|
| 9. |
Review
documentation of test strategies, plans, scripts and test populations
being used to validate the system functions as intended. Evaluate
whether strategies and plans are addressing key risk areas in
a reasonable and complete manner. Evaluate the processes ability
to be quickly re-performed when problems do exist. Evaluate
plans for documenting test results so that causes of problems
can be quickly analyzed and corrected. Evaluate whether testing
results documentation will effectively support management decision
to cut over to the system. |
|
|
|
| 10. |
Obtain
information on strategies to convert data from the legacy
system to the new system. Obtain information on plans to validate
and clean data before being converted. Obtain information
on controls being put in place to assure data corrected before
conversion will not be re-corrupted prior to cut over to the
new system. Evaluate the strategy's capability to assure the
data is accurate and complete. Identify risk points in the
conversion process and controls planned by the project team
to minimize those risk.
Some
specific risks to consider are:
- data
being converted comes from multiple sources which contain
different data values,
- data
on the legacy system is known to be partially corrupt,
- data
required in some fields of the new system does not currently
exist,
- data
is lost or corrupted in conversion process,
- conversion
requires a larger processing window than the business process
can permit,
- transactions
processed during the conversion period is lost or duplicated
on the new system, and
- no
tools exist to validate integrity of the data being converted
(i.e., that it is complete and accurate).
|
|
|
|
| 11. |
Obtain
a list of interfaces and a list of interface controls to ensure
that significant interface between the _________and other networks/computers
(e.g. BASIS mainframe) is appropriate. (i.e. is transferred
data authorized, complete and accurate?). Document how files
are transferred. Document how the users assure business transactions
transferred are authorized before being transferred. Document
reconciliation procedures and use of other tools between the
application and other systems to help ensure completeness and
accuracy. Evaluate effectiveness of interface controls. |
|
|
|
| 12. |
Obtain
information on rollout plans and definitions of roles and responsibilities
in the new system. Review sample system documentation. Evaluate
adequacy of rollout plans (i.e., processes to assure successful
cutover). |
|
|
|
| 13. |
Obtain
information on contingency plans and the trigger dates for
beginning preparations to place those plans in place. Evaluate
what risks have not yet been evaluated or do not have an effective
plan to mitigate risks. Evaluate if top management is effectively
informed of the impact of the risk and has made a strategic
decision to accept the risk. Evaluate the reasonableness of
those decisions.
Some
specific risks to consider are:
- development
problems occur causing the system implementation to be delayed,
- equipment
problems exist causing the system implementation to be delayed,
- the
data conversion fails causing system implementation to be
delayed,
- the
data conversion/ cut over to the new system takes longer
than expected (e.g., a week instead of a day),
- the
system does not run as fast as expected or needed,
- the
system is implemented with missing functionality, and
- the
system is implemented and does not function as intended.
|
|
|
|
| |
Review
of processes to manage system performance and batch processing
issues |
|
|
|
| 1. |
Obtain
information on processes being developed to assure batch processes
are appropriately scheduled and controlled. Evaluate tools and
procedures that are being put in place to assure consistent
and timely processing of transactions that need to be run in
a specific sequence or time frame. Evaluate processes being
put in place to assure transaction processing is not lost, duplicated
or run at the wrong time. Evaluate processes being put in place
to assure an adequate batch is window is available for necessary
processing. |
|
|
|
| 2. |
Obtain
information on processes being put in place to:
- define
acceptable system performance,
- identify
when performance problems are occurring,
- implement
appropriate corrective measures in a timely basis.
Evaluate
if the process is reasonable and complete. Evaluate if the
process will be able to effectively project future system
enhancements.
|
|
|
|
| |
Review
program change management process and tools associated with
development of the new system |
|
|
|
| 1. |
Background
note:
Controls
should ensure that program changes are properly designed,
tested, approved, incorporated within the application and
applied to the production environment. Controls should ensure
that change requests are handled appropriately.
|
|
|
|
| 2. |
Document
the client's program change process. Evaluate the process
for appropriate segregation of duties to the program change
process has integrity. Traditionally the following steps occur:
- User
authorizes specifications for development and documents
approval,
- Programmer
copies program to test environment,
- Programmer
modifies program,
- Programmer
Compiles and links program modules,
- Programmer
tests the program,
- Program
notifies user that program is ready for formal testing by
the user,
- Program
change coordinator (someone other than programmer) copies
the program to a test environment,
- User
tests the program in a test environment,
- User
approves changes & documents approval, and
- Program
change coordinator moves program to production.
|
- |
- |
- |
| 3. |
Obtain
sample documentation supporting the move process. Evaluate
how the application owner is assured:
- Only
authorized changes are moved into production,
- Source
and executable code are in sync,
- All
necessary program links are completed,
- Source
code is not lost,
- Code
changes can be identified, and
- Program
changes can be rolled back if not functioning as intended.
|
|
|
|
| 4. |
Obtain
information on how changes are managed so effort is not lost
if changes are being made to multi-versions of the system at
the same time. Evaluate risk associated with the change process. |
|
|
|
| 5. |
Evaluate
tools and processes to prevent and identify possible corruption
of program code when placed in production. |
|
|
|
| 6. |
Obtain
information on plans for processing emergency changes. Evaluate
how integrity risks are mitigated. |
|
|
|
| 7. |
Obtain
information on how access rights are currently configured and
what is planned when the system is placed in production. Evaluate
risks to the integrity of the program change process with current
and planned configuration. |
|
|
|
| |
Review
of access controls |
|
|
|
| 1. |
Evaluate
the security of the _________application and the operating environment
that the application resides in. Evaluate if security provide
adequate controls to ensure the application programs and data
are not accessed, modified or deleted by unauthorized individuals. |
|
|
|
| 2. |
Obtain
information on how authentication and intrusion detection performed.
Evaluate adequacy of process. |
|
|
|
| 3. |
Obtain
information on process for assigning access rights. Obtain list
of users and access rights. Evaluate reasonableness of rights. |
|
|
|
| 4. |
Obtain
information on procedures used to harden operating system. Evaluate
adequacy of process. |
|
|
|
| |
Review
of backup and recovery procedures |
|
|
|
| 1. |
Obtain
information on how program libraries (data and program files)
are backed up. Evaluate if process minimizes risk of development
effort. |
|
|
|
| 2. |
Document
file backup controls to ensure that only the most current and
authorized versions are used as the basis for future maintenance
and disaster recovery. Evaluate if reasonable controls to ensure
integrity of application and data. |
|
|
|
| 3. |
Obtain
continuity plans and with information gathered in step 1 of
System Development Approach, evaluate risk of business failure
in the event of a disaster. |
|
|
|
Return
to: Audits : U
of M Home
