Accelerating software development with generative AI for the public sector

Public sector software development faces challenges like strict security demands, compliance complexity, and legacy system maintenance. According to Unlocking the potential of public-sector IT projects, a study by McKinsey & Company: “Public-sector IT projects are more likely to miss their marks compared with private-sector projects—and at a greater cost. More than 80 percent of public-sector IT projects overran their schedules.”

Generative AI can automate tasks like code generation, testing, and documentation, but public sector customers must prioritize solutions that meet unique regulatory and operational requirements. Addressing these challenges with a secure and compliant generative AI solution can transform software development, helping organizations deliver projects on time while reducing costs.

This post outlines the considerations and solutions to evaluate when selecting an AI coding solution.

Considerations for adopting AI for software development

Choosing the right AI solution for software development requires careful consideration of your organization’s unique needs, priorities, and constraints. This section provides a framework to guide your decision-making process, empowering you to evaluate key factors such as model selection, compliance requirements, or data privacy in order to make informed choices that align with your organization’s requirements.

SLDC use cases

The most important consideration is how AI can be used in each phase of the software development lifecycle (SDLC), and in which phase AI would be most beneficial for your organization. Consider the following phases and how AI can be helpful during them:

1. Plan and design

1. • Understanding existing applications
   • Generating architectures based on project requirements
2. Implement
   • Real-time code autocompletion
   • Vulnerability detection
   • Legacy code modernization (for example, Java or .NET migrations)
3. Testing
   • Automated unit test generation
   • Synthetic data creation
4. Deployment
   • Automating continuous integration and continuous deployment (CI/CD) pipelines
   • Generating deployment environment configurations
5. Maintain
   • Performance optimization alerts
   • Security patch recommendations

Deployment options

Different deployment options offer varying levels of flexibility, control, and compliance. These deployment options include:

• Software-as-a-service (SaaS): The service provider owns and operates the AI stack with options to support multi-tenant or single-tenant architecture. This option provides a simple way to get started with no underlying infrastructure to manage or secure but with less flexibility and control.
• Self-managed: AI infrastructure and software stack are owned and operated within customers’ infrastructure either in the cloud or on premises. This option typically offers the most flexibility and control, but requires administrative overhead and additional infrastructure costs especially if self-hosting larger models.
• Hybrid: This option is a mix of the two models, where some components of the platform are hosted within a service providers’ infrastructure (for example model hosting), and other components such as Retrieval Augmented Generation (RAG) and development usage analytics remain within the customers’ environment. This approach can offer greater control of the data, but can present more complex integration or compliance challenges.

Model selection

Customers should consider which models are available for each deployment option and if control over model section is important. Open source models offer transparency for audits and align with federal approval processes, but might require more technical resources to implement if self-managed. Proprietary models can offer unique or differentiating capabilities with vendor support, but with additional costs. Other considerations are fine-tuning support to improve model performance on domain-specific tasks and understanding how the model was trained and datasets used for AI transparency and quality of outputs. Refer to AWS AI Service Cards to better understand Amazon Web Services (AWS) AI services and models.

Context awareness

Consider whether your use case requires your AI models to have context awareness of your organization’s internal libraries, proprietary algorithmic techniques, and enterprise code style. You can use RAG techniques to enhance your AI models with these internal knowledge bases to enable quick and efficient retrieval of context-relevant information without leaving your working environment and disrupting your workflow.

Data privacy

Weigh whether your selected model or service provider can use or store content for service improvement and whether there is an option to opt out. Some service providers have agreements with model providers to be opted out by default. As described in AI services opt-out policies in the AWS documentation, AWS provides the option to “create opt-out policies for an individual AI service, or for all services supported by AI services opt-out policies.”

Compliance

When selecting a deployment method and model, take into account your unique workload’s compliance requirements. For federal use cases, observe compliance requirements such as Federal Risk and Authorization Management Program (FedRAMP) and Department of Defense (DoD) Cloud Computing (CC) Security Requirements Guide (SRG) Impact Level compliance, or United States International Traffic in Arms Regulations (ITAR) compatibility. Other compliance requirements to consider include General Data Protection Regulation (GDPR), AWS System and Organization Controls (SOC), and Health Insurance Portability and Accountability Act (HIPAA).

Service terms and legal protections

Review licensing agreements and terms of use for both open-source and proprietary models. Consider protections against copyright intellectual property (IP) infringement claims on data generated by a foundation model (FM) being attributed to another company’s proprietary data. Refer to AWS Service Terms for AWS terms and agreements. Ultimately, the customer will be responsible to determine the legal liabilities for code generated by AI, including defects and vulnerabilities.

Ecosystem integration

User experience is an important factor to consider when using AI for software development. Developers can get the most immediate value out of AI when their AI deployment can integrate with their existing integrated development environment (IDE) or DevOps tools. For example, Amazon Q Developer has built-in integrations with AWS services that provide their own coding environments, such as Amazon SageMaker Studio, AWS Glue Studio, and AWS Lambda. Other services or third-party partners such as Gitlab Duo integrates with Amazon Q. When choosing a self-managed deployment on AWS, you can use open source plugins such as Continue, Cline, and Aider.chat in order to integrate your AI coding assistant deployment with your organization’s preferred development tools.

Pricing and cost

Different sized teams and usage patterns can benefit from different pricing models depending on deployment option. Charges can be per user, per token, per compute, or other factors. Also consider the cost of direct or indirect labor to build and maintain your self-hosted AI systems and the human capital and expertise required to navigate complex public sector requirements.

Support

Different deployment options also offer varying levels and types of support. Software vendors, model providers, and cloud service providers all have separate support structures. Open source software and model support is often community driven, with separate providers offering enterprise support or professional services.

AI-powered software development options for the public sector

Determine which considerations are most important for your AI code development, then evaluate available solutions and technologies to make an informed decision. Today, several options exist for public sector customers when evaluating AI coding assistant solutions including:

• Amazon Q Developer: Amazon Q Developer is the most capable fully managed AI-powered assistant for software development that reimagines the experience across the entire software development lifecycle. For example, Amazon Q Developer Transform capabilities can accelerate modernization and migrations of Java, .NET, Mainframe, and VMWare workloads.
• AWS Partner Solutions: There are emerging independent software vendor (ISV) partners that offer solutions with different capabilities and deployment options that align to public sector needs. For example, Gitlab Duo offers self-hosted options where software components can run on your own infrastructure. Another partner, Windsurf (formerly Codeium), offers fully managed options that are FedRAMP and DoD CC SRG Impact Level authorized. Refer to the AWS Partner Network (APN) for additional partner solutions.
• Build it yourself: You can also build your own AI coding assistant on AWS using AWS AI services and solutions with open source AI development tools. This approach is ideal for customers that need maximum flexibility and control integrating the latest AI capabilities with their software development environments or to meet specific security or compliance requirements. For guidance and patterns on this option, refer to Building an AI coding assistant on AWS: A guide for federal agencies.

Conclusion

Choosing the right AI solution for software development depends on your organization’s unique needs, including compliance standards and deployment preferences. By considering factors such as the most impactful SDLC phase in which to integrate AI, deployment models, model selection, and compliance needs, you can craft a solution that aligns with your organization’s goals and constraints.

To quickly get started exploring innovative ways of accelerating software development, individual users can get started with Amazon Q Developer in the AWS Management Console, AWS Command Line Interface (AWS CLI), or in their IDE on the perpetual Free Tier.