Program of Measurement and Evaluation

Name : Anggoro Boedi Riyanto
Student Number : C1L011025
PERFORMANCE MEASREMEN! AN" E#ALA!$ON
ABS!RAC!
Program evaluation is a rich and varied combination of theory and practice. It is widely
used in public, nonprofit, and private sector organizations to create information for planning,
designing, implementing, and assessing the results of our efforts to address and solve problems
when we design and implement policies and programs. Evaluation can be viewed as a structured
process that creates and synthesizes information intended to reduce the level of uncertainty for
decision makers and stakeholders about a given program or policy. It is usually intended to
answer questions or test hypotheses, the results of which are then incorporated into the
information bases used by those who have a stake in the program or policy. Evaluations can also
discover unintended effects of programs and policies, which can affect overall assessments of
programs or policies.
$N!RO"C!$ON
Many analytic approaches have been employed over the years by the agencies and others
to assess the operations and results of federal programs, policies, activities, and organizations.
Periodically, individual audit and evaluation studies are designed to answer specific questions
about how well a program is working, and thus such studies may take several forms. he !esults
"ct e#plicitly recognizes and encourages a complementary role for these types of program
assessment$ annual performance reports are to include both performance measurement results
and evaluation findings. his document describes and e#plains the relationships between
performance, measures, and evaluation.
CON!EN!
"e%inition&
Performance measurement is the ongoing monitoring and reporting of program
accomplishments and progress toward preestablished goals. %or many programs, requirements
can be met through performance measurement, which includes collecting data on the level and
type of activities &inputs' and the direct products and services delivered by the program
&outputs'.
Program evaluation is a systematic process of obtaining information to be used to assess
and improve a program. In general, organizations use program evaluations to distinguish
successful program efforts from ineffective program activities and services and to revise e#isting
programs to achieve successful results. (onducting evaluations is an integral part of operating
and managing a program because it helps to determine whether you are meeting the needs of
your client base. he type and application of program evaluation methods depend on the mission
and goals of the program.
Most performance measures can be grouped into one of the following si# general
categories.
• Effectiveness$ " process characteristic indicating the degree to which the process output
&work product' conforms to requirements.
• Efficiency$ " process characteristic indicating the degree to which the process produces
the required output at minimum resource cost.
• )uality$ he degree to which a product or service meets customer requirements and
e#pectations.
• imeliness$ Measures whether a unit of work was done correctly and on time. (riteria
must be established to define what constitutes timeliness for a given unit of work. he
criterion is usually based on customer requirements.
• Productivity$ he value added by the process divided by the value of the labor and capital
consumed.
• *afety$ Measures the overall health of the organization and the working environment of
its employees.
+isted below are seven important benefits of measurements$
• o identify whether we are meeting customer requirements. ,ow do we know that we are
providing the services-products that our customers require.
• o help us understand our processes. o confirm what we know or reveal what we don/t
know. 0o we know where the problems are.
• o ensure decisions are based on fact, not on emotion. "re our decisions based upon well
documented facts and figures or on intuition and gut feelings.
• o show where improvements need to be made. 1here can we do better. ,ow can we
improve.
• o show if improvements actually happened. 0o we have a clear picture.
• o reveal problems that bias, emotion, and longevity cover up. If we have been doing our
2ob for a long time without measurements, we might assume incorrectly that things are
going well. &hey may or may not be, but without measurements there is no way to tell.'
• o identify whether suppliers are meeting our requirements. 0o our suppliers know if our
requirements are being met.
Performance measures may address the type or level of program activities conducted
&process', the direct products and services delivered by a program , or the results of those
products and services. " 3program4 may be any activity, pro2ect, function, or policy that has an
identifiable purpose or set of ob2ectives.
Program evaluations are individual systematic studies conducted periodically or on an ad
hoc basis to assess how well a program is working. hey are often conducted by e#perts e#ternal
to the program, either inside or outside the agency, as well as by program managers. " program
evaluation typically e#amines achievement of program ob2ectives in the conte#t of other aspects
of program performance or in the conte#t in which it occurs. %our main types can be identified,
all of which use measures of program performance, along with other information, to learn the
benefits of a program or how to improve it.
Evaluation as a field has been transformed in the past 56 years by the broad7based
movement in public and nonprofit organizations to construct and implement systems that
measure program and organizational performance. 8ften, governments or boards of directors
have embraced the idea that increased accountability is a good thing, and have mandated
performance measurement to that end. Measuring performance is often accompanied by
requirements to publicly report performance results for programs.
Program evaluation and performance measurement complement each other in$
a. 0eveloping or improving measures of program performance.
b. ,elping to understand the relationship between activities and results.
c. 9enerating data on program results not regularly collected or available through other
means.
d. Ensuring the quality of regularly collected performance data.
Re'ation&(i) bet*een Per%orman+e Mea&urement and Program E,a'uation
Performance measurement focuses on whether a program has achieved its ob2ectives,
e#pressed as measurable performance standards. Program evaluations typically e#amine a
broader range of information on program performance and its conte#t than is feasible to monitor
on an ongoing basis.
0epending on their focus, evaluations may e#amine aspects of program operations &such
as in a process evaluation' or factors in the program environment that may contribute to its
success, to help e#plain the linkages between program inputs, activities, outputs, and outcomes.
"lternatively, evaluations may assess the program:s effects beyond its intended ob2ectives, or
estimate what would have occurred in the absence of the program, in order to assess the
program:s net impact. "dditionally, program evaluations may systematically compare the
effectiveness of alternative programs aimed at the same ob2ective.
;oth forms of assessment aim to support resource allocation and other policy decisions to
improve service delivery and program effectiveness. ;ut performance measurement, because of
its ongoing nature, can serve as an early warning system to management and as a vehicle for
improving accountability to the public.
" program evaluation:s typically more in7depth e#amination of program performance
and conte#t allows for an overall assessment of whether the program works and identification of
ad2ustments that may improve its results.
Pro+e&& -or $m)'ementation. E,a'uation
his form of evaluation assesses the e#tent to which a program is operating as it was
intended. It typically assesses program activities: conformance to statutory and regulatory
requirements, program design, and professional standards or customer e#pectations.
Out+ome E,a'uation
his form of evaluation assesses the e#tent to which a program achieves its outcome7
oriented ob2ectives. It focuses on outputs and outcomes &including unintended effects' to 2udge
program effectiveness but may also assess program process to understand how outcomes are
produced.
$m)a+t e,a'uation
Impact evaluation is a form of outcome evaluation that assesses the net effect of a
program by comparing program outcomes with an estimate of what would have happened in the
absence of the program. his form of evaluation is employed when e#ternal factors are known to
influence the program:s outcomes, in order to isolate the program:s contribution to achievement
of its ob2ectives.
Co&t/Bene%it and Co&t/E%%e+ti,ene&&
"nalyses these analyses compare a program:s outputs or outcomes with the costs
&resources e#pended' to produce them. 1hen applied to e#isting programs, they are also
considered a form of program evaluation. (ost7effectiveness analysis assesses the cost of
meeting a single goal or ob2ective and can be used to identify the least costly alternative for
meeting that goal. (ost7benefit analysis aims to identify all relevant costs and benefits, usually
e#pressed in dollar terms.
Ca&e Study
In (anada, a ma2or update of the federal government:s evaluation policy was announced
in 566< &;*, 566<'. he main plank in that policy is a requirement that federal departments and
agencies evaluate all their programs on a =7year cycle. Program evaluation is e#plicitly linked to
assessing 3program performance4>what is noteworthy is that performance includes the
economy, efficiency, and effectiveness of programs. %or the first time, the performance
measurement function in all departments and agencies, which had been a separate management
activity, is now linked to the evaluation function. ,eads of departmental evaluation units are
e#pected to take some responsibility for ensuring that program performance measures are
implemented in ways that support program evaluation requirements.
Performance management is now central to public and nonprofit management. 1hat was once an
innovation in the public and nonprofit sectors in the early ?<<6s has since become an
e#pectation. %undamental to performance management is the importance of program and policy
performance results being collected, analyzed, compared &often with performance targets', and
then used to monitor and make decisions. Performance results are also e#pected to be used to
increase the transparency and accountability of public and nonprofit organizations and even
governments, principally through periodic public performance reporting. Many 2urisdictions
have embraced mandatory public performance reporting as a visible sign of their commitment to
improved accountability.
In the @nited *tates, successive federal administrations beginning with the (linton
administration have embraced program goal setting, performance measurement, and reporting as
a regular feature of program accountability. he ;ush administration between 5665 and 566<
emphasized the importance of program performance in the budgeting process. he 8M;
introduced assessments of departments and agencies using a methodology called P"!
&Performance "ssessment !ating ool'. Essentially, 8M; analysts reviewed e#isting
evaluations conducted by departments and agencies as well as performance measurement results
and offered their own overall rating of program performance. Each year, one fifth of all federal
programs were 3P"!ed,4 and the review results were included with the administration:s budget
request to (ongress.
he 8bama administration, although departing from top7down P"! assessments of
program performance , continued this emphasis on performance by appointing the first %ederal
(hief Performance 8fficer, leading the 3management side of 8M;,4 which is e#pected to work
with agencies to 3encourage use and communication of performance information and to improve
results and transparency4 &8M;, 56?5'. "lso evident is the emphasis on program evaluation as
an approach to assessing performance. In the fiscal year 56?? budget cycle, for e#ample, a total
of AB high7profile evaluations of programs were approved for funding for ?C departments and
agencies .
CONCLS$ON
1hile performance measures can be described only what is occurring in your program,
program evaluation provides you with an overall assessment of whether the program is working
and can help identify ad2ustments that may improve the program results. Performance
measurement data can be used to detect problems early in the process.
REFERENCE
www.ovcttac.gov
@nited *tates 9eneral "ccounting 8ffice &9"8' 9lossary