SOX Compliance Solution Implementation Outcomes

Having completed a full controls test management solution implementation from inception to user rollout in approximately 30 weeks, KBR reports an overall high level of satisfaction. This assessment is derived from reports with respect to a number of outcomes related to implementation processes and objectives, including:

Timing: KBR completed its SOX compliance solution deployment and hit a go-live date that was delayed only one week from the target date. Given the aggressive timeline and lack of substantive impact on supported business operations, KBR considered the delay to be well within the margin of success.
User Adoption and Acceptance: KBR reports 100% adoption at go-live, as use of the solution became required to perform controls test and review tasks. However, business users reported high satisfaction with the new SOX compliance solution and reported no major issues, bugs, or errors that interfered with their ability to use the tool. In a few rare cases, users indicated a reluctance to adopt the new process. These were addressed by Financial Controls via one-on-one walkthroughs.
Business User Impact: Business satisfaction reported ranges from “Excellent” to “Extremely High”. KBR received no negative comments about system functionality or usability. Users reported increased ease of use and reduced errors or confusion resulting from the increased usability and clarity provided by the DoubleCheck platform compared to the prior, dual-system model. KBR indicated that this resulted in improved efficiency and time savings in data entry compared to the prior solution.
SOX Controls Assessment Administration and Management Impact: KBR reported satisfaction with the DoubleCheck system’s impact on SOX controls assessment as “very high”. The organization identified some minor change in the effort required to make system changes and maintain cleanliness of data in the new system versus the legacy system. KBR reports that this change is a consequence of added complexity resulting from the flexibility of the new system and the addition of features such as the automated workflow, increase in data properties captured, and enhanced reporting capabilities. However, the organization finds that the impact is minimal, and “well worth” the added capabilities of the system.
Report Creation and Consumption: Satisfaction with the DoubleCheck platform’s reporting capabilities is “very high”. KBR reports that significantly more information can be retrieved by users, with greater flexibility and accuracy, than previously. Report creation is faster, easier to use, and pulls from a wider range of data than previously available. KBR reports that this has almost eliminated manual report creation activities. The system now runs certain reports automatically every weekend with no manual involvement by the Financial Controls Group and distributes them to designated users. Report production frequency and self-service access have also increased, as more ad hoc reports are being created, analyzed, and used for status reporting. Finally, the organization reports increased use of data visualization aids, such as scorecards and dashboards to support executive consumption.

“User satisfaction is extremely high. Users were elated with the new application because the prior solution was very inefficient and cumbersome to use. It required duplicate entry into two systems, and neither system had all the data that a user needed to see at one time. Almost all reports were manual prior to DoubleCheck, as well, and those that were not were very limited, partly because the data captured in the system was limited. We are able to report much more information from the new system.”

Patricia Pavlick
Project Manager, Financial Controls, KBR

As noted above, the 7.5-month implementation cycle reported by KBR ranks well within the Best-Case implementation scenarios benchmarked in Blue Hill Research’s July 2015 Benchmark Report Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios (Table 5).

In terms of technical deployment alone, KBR accomplished its core operations within fourteen weeks, with only one week of delay beyond its planned go-live date. This closely matches the experiences of Best-Case implementation scenarios, where technical deployment time falls between three and four months. By contrast, Worst-Case implementation scenarios benchmarked by Blue Hill Research involved technical deployment times that fell between 11 and 16 months in length, which well exceed the total implementation cycle reported by KBR. In addition, the levels of user satisfaction reported by KBR match or exceed those reported in Best-Case scenarios.

Blue Hill Analysis: Implementation Best Practices Demonstrated

KBR’s ability to achieve an implementation within a time cycle and at levels of organizational satisfaction that match the Best-Case Scenario benchmarked by Blue Hill resulted from a number of aspects of the organization’s approach. These aspects include, among others:

The level of detail and connection to business and operational needs employed in requirements planning
The use of requirements to guide all subsequent stages of the investment and implementation process
A rigorous approach to project management throughout deployment
Selection of an offering whose architectural characteristics supported implementation goals and a vendor that prioritized the same goals

Many of the practices demonstrated by KBR echo practices employed in Best-Case implementations identified in Blue Hill Research’s July 2015 Benchmark Report Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios.

Practices identified in this report include:

An implementation scope focused on support for established business operations, rather than using the software investment as an agent of process change
Primary responsibility for the implementation rests with the functional line of business owner
IT stakeholders are involved at early stages of solution, evaluation and investment planning
A preference for configurable applications driven by attention to both present and anticipated change to application needs

Blue Hill’s assessment of KBR’s strategic decisions, methodology, and outcomes reveal additional contributing factors that resulted in the success of its implementation. Table 6 summarizes these contributing factors as they related to organizational components, strategic planning considerations, operational excellence, and technical aspects of the DoubleCheck solution itself.

Based on its analysis of these factors, Blue Hill has identified six best practices in the organization’s implementation that directly contributed to the results realized, which are summarized in the following sections.

Define Precise And Complete Business Requirements

Arguably the most important step taken by KBR was the creation of detailed, specific, and written business requirements.

At every stage of the implementation process, the 75 business requirements defined by the vendor played a role in narrowing attention to the underlying operational needs driving the investment. As such, these requirements became KBR’s primary tool for solution selection as well as implementation planning. In the former, the organization’s requirements document helped to define its RFP questionnaire as well as its demo script and evaluation framework. In defining the solution itself, the requirements document influenced the shape of KBR’s configuration specifications as well as its UAT test plans. The requirements document even assisted in KBR’s efforts at user role definition, workflow design, and data property models, all factors that are often left to deployment stages and can substantially slow the implementation. While KBR did not have these elements of the solution complete prior to implementation, the early identification of requirements and processes provided an established framework that anticipated much of the effort involved in this later stage.

At each stage, KBR was able to take advantage of prior efforts to identify both the functionality it needed as well as the non-functional architectural and delivery methods that would permit it to effectively achieve its goals. Its ability to quickly work through vendor selection and to adhere to an ambitious deployment cycle also rolled out, directly and indirectly, from the adherence to clear and unchanging requirements.

From this standpoint, it is important to recognize that the technology investment was not a transformation vehicle. Rather, KBR had already analyzed its SOX controls test and review processes and made the strategic changes it identified. The technology layer was implemented solely as an enabler to those changes. This approach is consistent to the implementation planning practices highlighted among Best-Case implementations in Blue Hill’s Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios Benchmark Report.

Companies planning similar investments will benefit from modeling their own approaches on KBR’s requirements planning process. The organization dedicated significant time to this aspect of the implementation (approximately one month), prior to any major review of software options. Special note should be taken of KBR’s “no surprises” approach, which attempted to identify factors that would impact the success of the implementation. The impact of this step on subsequent activities cannot be overemphasized.

“If your requirements are really precise, the rest flows from there. You’ll have a roadmap that you will use all the way through the implementation. I recommend creating detailed, specific written business requirements that identify all the data elements you want to capture and the values you need for each. Everything else starts from there. You’ll have your roadmap for everything from system selection to design to acceptance testing.”

Patricia Pavlick
Project Manager, Financial Controls
KBR

Locate Primary Implementation Responsibility With The Business Operation Owner And Seek Guidance From Other Stakeholders Early

A recurring theme throughout KBR’s implementation was the identification of the Financial Controls Group as the primary owner of the program. The Financial Controls Group possesses the authority and responsibility for SOX controls assessment and thus represents the natural owner for determining the solution requirements that would best support those operations.

Often, a functional line of business owner will be only one member of a committee responsible for an implementation. In other scenarios observed by Blue Hill, other stakeholders are brought in last minute when approvals are needed, rather than at times when they have the ability to influence planning or strategy. For example, consider situations where line of business stakeholders identify a desired solution and then attempt to “farm it out” to IT or other departments that may understand the organization’s technological ecosystem but lack deep insight into the business processes to be supported.

By contrast, KBR primarily located implementation decision making and execution with a single team: Financial Controls. The Financial Controls Group held primary responsibility for a wide array of activities, including business requirements definition, solution evaluation, project management, user acceptance management, training, rollout strategy, and other activities. Where other stakeholders were involved, it was most often to provide guidance on the decision, rather than to take responsibility for elements of decision making or execution of implementation activities. It should be noted that involvement of outside stakeholders, be it business users, KBR’s Chief Financial Officer, or IT, occurred early in the process to set constraints or guidelines, such as budget or technical requirements. This approach effectively served to empower the Financial Controls Group to act within boundaries that served the larger organizational needs, while preserving its ability to identify a solution that would best fit its business operations.

As noted above, Blue Hill Research identified this “early advisory” approach as a key practice in Best-Case implementations in the Contributors to GRC Implementation Success: Avoiding Worst-Case Scenarios Benchmark Report. However, for a variety of reasons, the exact approach employed may not fit all organizations. Organizational boundaries or procurement policies may require the use of a different process. Alternatively, aspects of the solution or process change considered may insert limitations. For example, if the organization does not opt for a vendor-hosted offering, considerably more involvement from IT will be required. Whatever the options available, however, early solicitation of guidance and alignment of requirements to organizational constraints will play a key role in effective scoping of the implementation.

“We were trying to plan a new system for our users. It was important for us to have a simplified, user-friendly approach to the system interface and our workflows. We have hundreds of users and frequent changes in those user roles. The system needed to be intuitive and capable of being picked up by a new user quickly and without extensive training. Both early in the process and toward its end, we involved key business users to understand their needs. That went a long way to getting us to the right place.”

Steve Vontur
Director of SEC Reporting and Financial Controls, KBR

Secure Executive-Level Support And Championship For The Project

In addition to obtaining guidance from key stakeholders in early stages of its investment and implementation planning, KBR also secured executive-level support from the CAO. In part, support of the CAO, along with the CFO, represented a gating requirement for the initial investment approval. However, the involvement of the CAO in this implementation went deeper. The CAO worked at early stages of the process to help align and gain support for the investment business case. Throughout the process, the CAO also helped to support the implementation by helping to identify strategic needs and prioritization as well as provide feedback on data elements and workflow components to be addressed by the DoubleCheck controls management platform.

The involvement of an executive champion is a frequent component of a successful implementation of a major enterprise technology investment. The role taken by the CAO here provides a prime example of the impact that executive stewardship provides, by securing support and buy-in from other organizational stakeholders and helping to define the importance of the investment to the business. Continuing involvement also helps to maintain focus on the implementation effort.

This is particularly important where an implementation is led by a business operation owner, as momentum for the implementation effort is often at risk of taking a backseat to the owner’s primary responsibilities. Involvement of the executive champion thus often assists in ensuring that implementation activities are accomplished “to spec” and on schedule.

Identify The Solution And Vendor Delivery Options That Align To Your Priorities

Several contributors to KBR’s success resulted from characteristics of the solution and deployment model that they selected. Similarly, the ability to successfully operate to a tight implementation schedule was a result of the selection of a vendor that was able to align on project goals and to collaborate to meet them.

The DoubleCheck GRC platform selected by KBR is vendor-hosted and rests on a highly configurable application architecture. These factors alone removed two major time investments of an implementation: the application environment provisioning and the customization of the application. Blue Hill Research’s August 2015 Solution Landscape GRC Vendor Implementation Success Strategies identified these factors as key vendor contributors to implementation success (see sidebar). Similarly, the selection of a vendor that provided a flat fee for implementation services and demonstrated a willingness to embrace KBR’s requirements and implementation plan helped to ensure that the implementation could be executed to plan without scope creep or unexpected delays.

Vendor Contributions to Implementation Success

Blue Hill Research’s August 2015 Solution Landscape GRC Vendor Implementation Success Strategies identified vendor and solution capabilities that correlated to Best-Case implementations, including:

Rapid Deployment Programs

Application Configurability

Availability of Out-of-the-box Functionality

Software as a Service (SaaS) or hosted delivery models

Subscription-based pricing

Blue Hill’s prior research has identified how these sorts of factors represent vendor supplied contributors to effective implementation. However, these characteristics are not ultimate guarantors of success in and of themselves. Solution attributes such as configurability or hosted deployment are factors to assess and trade off against other benefits and drawbacks of a solution. As such, KBR’s selection of these capabilities does not represent a best practice.

Rather, the best practice to be emulated is the organization’s attention to aligning solution architecture and delivery considerations to business and operational needs in planning the implementation. This has been mentioned above with reference to requirements definition. It bears repeating in the context of solution selection.

KBR identified its business requirements, operational preferences, and a go-live target intended to permit use of the platform to support its 2016 controls test review. It then identified non-functional attributes, such as the application architecture and hosting model, that would help further these needs. In so doing, KBR ensured that the solution and vendor that it identified possessed attributes that would facilitate its implementation objectives, rather than present compromises or challenges to work around.

Adopt A “Show Me” Approach To Vendor Claims

In addition to including factors with an impact on implementation, KBR demonstrated a strong emphasis in solution evaluation on ensuring that vendors provided concrete demonstrations of their ability to meet KBR’s needs. To this end, one may observe the emphasis that the organization put on vendors’ ability to demonstrate how it would address requirements in RFP answers and in the demo script and evaluation methodology. KBR disfavored pre-packaged presentations and sought concrete demonstrations of how the offering could address KBR’s requirements.

“We required the vendors we evaluated to provide specific responses regarding our business requirements, not only whether or not the system met them, but how. When you get an RFP response from a vendor, most give you ‘yes’ or ‘no’ for an answer. The truth is a vendor can often answer ‘yes’ honestly when the way that it is done makes you want to say ‘no’.”

Patricia Pavlick
Project Manager, Financial Controls, KBR

This “show me” approach plays a crucial role in ensuring that an organization is able to evaluate a vendor’s true capabilities and enabled KBR to gauge not only what functionality the vendor could effectively deliver, but also how much effort would be required to implement that functionality as a working solution for KBR. A solution that looks good in a “canned” demo or “checks all the boxes” in an RFP might require extensive work and application-level customization in order to adapt it to the buyer’s needs. Uncritical acceptance of vendor answers and a lack of attention to the concrete details of how a solution delivers its functionality will result in scope creep, compromises, and workarounds that retro-fit processes to the limitations of a tool. All of these outcomes work to lengthen deployment and erode value, sometimes by dramatic margins.

KBR found that the willingness and ability of DoubleCheck to configure its platform to meet its requirements within the demonstration provided a clear signal of the vendor’s ability to effectively accommodate its needs within its implementation window. In assessing other vendors, KBR observed an unwillingness to attempt to demonstrate the specific functionality required by KBR, as set forth in KBR’s demo script, which left open questions about whether the systems could be configured to KBR’s specifications quickly and easily. In this way, how a vendor responded to KBR’s requests provided qualitative indications regarding the potential working relationship with a vendor and limitations in its system. To this end, KBR observed that the insights it obtained into the DoubleCheck architecture and vendor relationship in the demo proved to represent key attributes that helped drive the speed and success of its deployment effort.

Once again, readers should observe how KBR’s development of detailed and complete requirements that attached to both functional and non-functional attributes of the solution permitted KBR to apply this level of scrutiny.

Without a clear articulation of its requirements, KBR could not assess how a vendor was able to track to these needs. Nor would a vendor have sufficient information with which to provide meaningful responses.

Adopt Formalized Project Management Practices To Manage The Implementation

No matter that the technical attributes of the DoubleCheck solution required less labor than a less configurable solution, KBR’s 3.5 month technical deployment cycle did not happen on its own. Rather, KBR identified a comprehensive project plan that accounted for all steps necessary to achieve its implementation goals and made effective use of standard project management practices to ensure that plan was followed.

Two major project management practices can be credited with the most impact of the operational effectiveness that KBR demonstrated in its technical deployment:

“Our approach was classic project management. You need a dedicated project manager, where the project is clearly that person’s first priority, on both the client and vendor side. You need to develop and use a detailed, task-level project plan and timeline. This is your project ‘bible’. You will adjust it when necessary, but it helps you to know what is needed to do and see where you need to adjust efforts to make your target.”

Patricia Pavlick
Project Manager, Financial Controls, KBR

The Project Plan and Schedule: The organization developed a detailed, task level project plan and timeline to serve as the project timeline and “bible” maintained by the Financial Controls Group. As at other stages, KBR’s prior definition of business requirements played a role, guiding the scope of the implementation and setting milestones based on objectives. It was reviewed weekly and kept up-to-date and evolved over time. In this way, changes to the project schedule served both to reflect changes and to help direct steps that were needed to keep the project on course. Use of the plan in this way ensured that deployment stakeholders maintained an eye on future tasks and prevented potential slippage and ensured that all parties remained aligned.
A Dedicated Project Manager: Rather than designate a professional project manager to the project, KBR assigned primary responsibility for project management to a member of the Financial Controls Group. This meant that the stakeholder responsible for moving the project forward did so with a clear sense of the business objectives and operational context involved. The designation of an internal stakeholder within the Financial Controls Group is particularly important as the individual was thus able to bring a thorough understanding of the business requirements that an outsider would not have been able to bring. As such, the project manager could both define the project plan and schedule and drive the execution and progression of tasks with close scrutiny and an eye toward the ultimate business objective. In addition, the project manager was able to work closely with the peer project leader at DoubleCheck in close partnership to drive a mutual focus on project dependencies, keeping pace with the project schedule, and communicating potential issues.

Organizations are unlikely to require extensive reminders of the value of good project management. However, several elements of KBR’s strategy deserve particular attention:

The rigorous attention by which the project schedule remained a living document and a planning resource, rather than a reporting instrument
Thorough understanding of underlying business operations and objectives provided context that guided the project management effort, accomplished by designating project management authority within the Financial Controls Group. (However, it should be noted that it is not a best practice to assign a functional owner with no project management experience)
Clear and transparent communication and partnership with vendor stakeholders, on a regular basis, was crucial to ensuring all parties remained on track

Key Observations and Takeaways

Within a 7.5 month cycle, KBR was able to progress from the identification of a solution investment need and developing a business case for the replacement of its legacy SOX controls system to running a live replacement. By any measure that Blue Hill has tracked, KBR’s implementation stands out as a model of success. This impression is echoed in the organization’s own assessments of its experiences and resulting satisfaction with its platform and its implementation experience.

The steps that enabled KBR to accomplish its objectives represent a mix of careful and rigorous planning, vendor evaluation, vendor partnership, and disciplined focus on business process and needs as well as familiar best practices related to executive sponsorship and project management. Organizations that look to KBR’s example to extract lessons for their own implementations will benefit from each of the best practices identified in this report. However, the importance of the development of precise and comprehensive business requirements cannot be understated. The organization’s effectiveness at all subsequent stages of the implementation resulted, directly or indirectly, from the efforts that KBR put into these requirements at the outset of its project. While Blue Hill has identified effective requirements generation as a best practice previously, the forethought and level of detail seen in KBR’s example is rare. While organizations are not guaranteed to see the same results in their own implementations, learning from these practices will help them to prevent issues related to scope creep, downstream compromises, and budget and schedule overruns that cause GRC implementations to become failures before any solution goes live.