AWS Partner Network (APN) Blog

The Journey to AWS MSP: Best Practices for Executing the AWS MSP Full Audit

Editor’s note: This is the second in a two-part series for AWS MSP Partners. Read Part 1 >>

By Marija Bulatovic, Global Leader, AWS MSP Program

The AWS Managed Service Provider (MSP) Partner Program recognizes AWS Advanced and Premier Consulting Partners that are highly skilled at providing full lifecycle solutions to customers.

An integral part of the AWS MSP journey is completing the full two-day audit with Information Security Systems International (ISSI), our third-party auditing firm and an AWS Select Consulting Partner.

Through the AWS MSP audit, partners demonstrate the business and technical best practices outlined in the AWS MSP Program Validation Checklist in a consistent and accurate way. This validation process assures our customers they are working with a credible, independent third-party audited AWS MSP Partner.

ISSI has been involved in hundreds of AWS MSP validation audits and has deep expertise working with cloud partners around the world.

In my first post in this two-part series, we had a Q&A with Chang Leong, Global Head of Auditing at ISSI, who shared best practices leading up to the AWS MSP audit.

In this post, we continue the conversation with Chang, who draws from a well of experience to provide advice on successfully executing the AWS MSP audit. We’ll focus on what to expect for the two-day-long AWS MSP audit—as well as post-audit activities for AWS MSP Partners.

AWS-MSP-Dark-Background

Q&A with ISSI

AWS: What can AWS Partners do to prepare for the AWS MSP full audit, and what can they expect during the audit?

Chang: First are some common-sense action items. Obviously, AWS Partners should start on time and make sure all stakeholders are available when needed. The audit takes two days, so align all calendars.

It’s not necessary to have every audit prep team member attend the entire audit. But everyone who was involved in the audit preparation, especially all subject matter experts (SMEs), should be on standby in case the auditor asks a question to ensure the expert on the topic is available.

It’s important that everyone presents their own evidence. Don’t present evidence on someone else’s behalf because you may not be able to answer it correctly during probing questions from the auditor. Remember that an audit is not just about checking off items on a list. The auditor wants to understand your processes, and will be looking for detailed, thorough responses.

AWS: How should partners prepare the evidence? Is it a good idea to create a comprehensive presentation?

Chang: We recommend that AWS Partners collate all of their evidence into a central folder, keeping track of which items were prepared by specific individuals, and organizing or tagging them with control numbers so they can be quickly retrieved when needed. The auditor may want to test your system by asking where you keep your records.

We do not recommend creating extensive PowerPoint decks for the audit. We have seen partners prepare hundreds of slides capturing every detail, including screenshots and embedded objects. They can spend a lot of time preparing them, but the result is often of little use.

A few slides are fine as a high-level guide for the discussion, but an audit is a systematic, independent, and documented process for obtaining objective evidence and evaluating it to determine the extent to which the audit criteria are fulfilled.

The audit involves sampling, checking, cross-checking, testing, and validating. The auditor will need to see the actual evidence, not screenshots or samples you prepared. The auditor will want to follow your steps and trace the evidence during a project—in other words, establish an audit trail.

AWS: What evidence should AWS Partners make sure they bring to the audit?

Chang: There are three key things partners need to bring to the audit. The first are the process documents required by the audit checklist, including documented procedures, process guides, instructions, and any related plans.

Next is the implementation evidence. This can include, but is not limited to, test reports, checklists, emails, minutes of meetings, design documents, and implementation plans.

Finally, there are the demonstrations. These typically are demos on the use of tools. In most cases, technology demonstrations must be done live in front of the auditor. You can use a test or sandbox environment for these demos.

AWS: Do you have advice on how to present evidence during an audit?

Chang: Start with an explanation of how you’re going to proceed. In this case, it’s fine to use just a few short slides to organize the discussion. Always describe your process first, even if a process description is not required by the checklist.

If you have a detailed written procedure, show it, but don’t dwell on the details in the procedure. Give a quick run-through. Then, briefly explain the content of the evidence.

It’s not necessary to dwell on details unless asked specific questions by the auditor. Watch for signals or gestures from the auditor that imply “too much,” “too little,” “too slow,” or “confusing.” If the auditor says “enough” and asks to move on, it means you have hit all of the points, so do not dwell on the subject. Don’t feel offended. It’s a good sign.

AWS: If the auditor says something you disagree with, is it appropriate to voice that disagreement?

Chang: Yes. State your position and reason, but do not argue. Sometimes there may be more than one way to understand a particular issue or process. You need to help the auditor understand how you did something. Even if it’s not done the usual way, it can be accepted as long as it meets the intent of the control.

AWS: Some partners fail the MSP audits. What are the main reasons for this happening?

Chang: In our experience, there are several general reasons why partners fail the audits. First, they are not able to provide the required evidence, such as processes, records, or documents.

Or, sometimes they will provide evidence, but it falls short because the partner had an incorrect understanding of what was required. Perhaps they were not paying close attention to the details of the requirements. The audit checklist is built on international and industry best practices, but in some regions there are practices that are not aligned with the checklist, which can lead to a failure in a particular part of the audit.

We also see partners fail because they have not properly reviewed and rehearsed their presentation, or they rushed through their prep as the audit date closed in. Sometimes failures happen when partners rely too heavily on a single SME during the audit prep, and that person isn’t there on the day of the audit.

AWS: How can partners avoid some of these pitfalls?

Chang: Partners can avoid pitfalls with a pre-assessment session, which is mandatory for partners going for the AWS MSP validation for the first time, though it’s optional for renewing partners. The pre-assessment auditor provides a review of the general audit process, helps identify process gaps and missing documents, and provides suggestions for remedial work.

The pre-assessment, which takes a full day, should be scheduled at least 4-6 weeks in advance of the audit. That provides enough time for any remedial work, which greatly increases a partner’s chance of a successful audit.

Note the pre-assessment is not designed to provide training, but to identify gaps. You can maximize the pre-assessment by preparing thoroughly, including gathering all necessary evidence. Use the session to get answers to your specific questions.

AWS: Once partners have gone through the audit, what do you recommend for post-audit activities?

Chang: Gather all stakeholders and conduct a comprehensive debrief. Identify follow-up action items and lessons learned. Do a post-mortem on any significant issues that were raised during the audit. If there’s an action item that needs a response, make sure to provide a corrective action, and not just a correction.

If a partner fails the audit, these action items are critical to getting back on track. If a partner passes, the audit team members and the organization as a whole should be proud of the effort and what it will do for the business going forward.

Auditors will also issue an “Opportunities for Improvement” document, or OFI, which contains suggestions, recommendations, best practices, and potential issues that appeared during the audit. Although the OFI recommendations are optional, we strongly encourage partners to act on them.

Finally, it’s a good idea to establish maintenance procedures to ensure the renewal of the AWS MSP designation—currently every three years—goes smoothly. Identify a champion in your organization to own and keep up to date on the requirements. The champion should direct compliance and ensure the system meets any requirement at any time.

If any of the members of the original preparation team leave the company, find replacements.

As a best practice, partners should conduct an annual internal review on the compliance, at least on the mandatory items against the most recent AWS checklist. In parallel, partners should be aware of the annual AWS MSP Performance-Based Renewal motion taking place between full audits designed to validate partner’s MSP practice on an ongoing basis.

Learn More About the AWS MSP Audit

We encourage AWS MSP Partners to continue to explore this blog post covering 5 Tips to Help Partners Prepare for the AWS MSP Audit:

  • Engage your AWS Partner team.
  • Organize a project team to drive preparation.
  • Carefully read (and understand) the AWS MSP Validation Checklist.
  • Score yourself fairly on the AWS MSP Self-Scoring Checklist.
  • Perform the pre-assessment.

Additional Resources

Here are some links about the AWS MSP Partner Program to help you:

.
ISSI-APN-Blog-CTA-1
.

ISSI – AWS Partner Spotlight

ISSI is an AWS Select Consulting Partner and consulting, auditing, and marketing company focused primarily on Managed Service Provider (MSP) channel partners.

Contact ISSI | Practice Overview

*Already worked with ISSI? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.