Managing AI Projects (In Large, Legacy-Driven Companies) - Part 2

Today’s edition of the Augmented Advantage is co-authored by Valentin Marguet, project lead in the automotive industry.

Hi there,

Managing AI projects, especially in large companies that have a big legacy, can be a huge pain. It requires good project management to make them work.

As we explored in Part 1: Setting Your AI Projects Up for Success, it’s all about finding the right mix of blending proven PM techniques with what’s needed to keep pace with AI’s rapid evolution.

Today, we discuss the next crucial phase: turning well-laid plans into actionable, high-impact steps. We'll explore six typical encounters that AI project managers will face sooner or later and strategies we use to overcome them.

Let’s explore these one by one!

Want to dive deeper into AI and finally take action to grow and innovate your business?

My AI Live workshops are a great place to start. The next one - The 5 AI Archetypes Unpacked (With Hands-On Examples) - is just around the corner:

6 Critical Encounters in the Life of an AI PM (And Essential Tools to Handle Them)

The following six encounters are very common in AI projects — they're common everywhere but hit especially hard in large, legacy-driven companies. These encounters are interconnected, often overlapping throughout your AI journey.

With Valentin, we used our experience to provide you with a set of tools and strategies to help you cope with these situations:

1. Making Make-or-Buy Decisions

Scenario: When implementing your AI project, choose between buying an off-the-shelf solution for quick deployment or custom-building one for your legacy systems. This decision will shape the project's trajectory from day one.

Why it’s important: In legacy companies, systems are deeply anchored; choosing the wrong path could lead to significant integration issues, project delays, and missed opportunities. Your choice needs to align with business goals while minimizing disruption.

How to approach it:

→ Apply a structured decision-making process and think in roadmaps. Building a prototype is one thing, but scaling out your AI solution to an entire organization is a different beast. Consider different stages of your use case and see which stage of the AI make-or-buy-decision process is most applicable to you.

Follow these steps:

1. Form a Cross-Functional Decision Team:

   • Include both AI enthusiasts and skeptics, led by a senior executive sponsor for swift alignment.

2. Conduct Option Analysis:

   • Create a weighted decision matrix focusing on integration, cost, time-to-value, customization, and scalability.

   • Score "buy" and "build" options. Tip: Refresh the old but bullet-proof SWOT tool here.

3. Assess Future Impact:

   • Evaluate options' effects on workflows, data governance, and roles.

   • Engage key users to balance short-term disruptions against long-term potential.

4. Make and Validate the Decision

   • Present recommendations to stakeholders, address concerns, and document the decision rationale.

   • Establish a clear implementation roadmap with integration milestones.

2. Managing AI Risks

Scenario: Concerns about potential risks in your AI project have reached senior management. They need a detailed plan to ensure that risks are identified, assessed, and mitigated before they can impact the project.

Why it’s important: In legacy companies, where processes are often rigid, proactively managing AI risks is essential to prevent disruptions and ensure smooth project execution.

How to approach it:

• Enhance Risk Register

  • Adapt FMEA methods to include AI-specific risks like data drift and model decay.

  • Create a living document with risk scores for likelihood, detection ease, and impact.

• Conduct AI Risk Workshops

  • Involve AI experts and legacy stakeholders in risk identification.

  • Use scenario planning to uncover potential AI failure modes.

• Develop Tiered Mitigation Strategies

  • Categorize risks into technical, operational, and strategic.

  • Create response plans with clear triggers and ownership.

• Implement Continuous Risk Monitoring

  • Set up automated alerts for key AI performance metrics.

  • Establish a regular risk review cycle aligned with sprints.

3. Fixing AI Issues Fast

Scenario: Your AI system is giving inconsistent results, causing concern among stakeholders. The pressure is on to diagnose and resolve these issues quickly before confidence in the project is lost.

Why it’s important: In legacy environments, where new technologies often face skepticism by default, unresolved AI issues can erode trust and threaten the entire project’s viability—especially at the very beginning.

How to approach it:

• Diagnose Issues Quickly:

  • Set up automated alerts for key metrics.

  • Use a rapid triage checklist for data, model, and integration.

  • Apply "5 Whys" for root cause analysis. Here’s a 5-Why GPT that might come in handy.

• Deploy a Rapid Response Team

  • Maintain a go-to list of 3-5 cross-functional experts.

  • Create a virtual "war room" with all necessary tools and contacts.

  • Set time-bound goals for diagnosis and action.

• Implement and Verify Quick Fixes

  • Identify corrective measures and verify results quickly, ideally within a week.

  • Balance speed and impact.

  • Test in a controlled environment.

• Fast-track User-Reported Issues:

  • Set up a priority AI issue reporting channel for quick feedback.

  • Develop a concise "AI Fix Brief" template to share with stakeholders.

4. Adapting AI to New Needs

Scenario: Midway through your AI project, the business requirements change, and the landscape shifts towards uncharted territory. You need to adapt quickly without derailing the entire project.

Why it’s important: Staying flexible in AI projects is key. While this often contradicts the strive for structure and predictability in legacy companies, the ability to pivot and adapt your AI project is crucial for maintaining its relevance and value.

How to approach it:

• Adopt a flexible project management approach

  • Adaptability isn't about surviving change—but thriving in it.

  • Make sure your PM accepts change as the rule, not the exception.

• Synchronize Development:

  • Divide work into 2-4 week sprints.

  • Align AI development with legacy update cycles.

  • Create a unified project board for visibility.

• Streamline Change Management:

  • Conduct regular sprint reviews.

  • Implement a quick change evaluation process.

  • Use a red/yellow/green system for rapid decisions.

• Phase Testing and Deployment

  • Start with isolated AI tests, then gradually integrate with legacy systems.

• Foster Continuous Feedback

  • Set up diverse input channels.

  • Make feedback collection routine.

  • Update priorities based on real-time insights.

5. Driving AI Adoption and User Engagement

Scenario: Your AI project is progressing well, but awareness, adoption, and trust across the company are lacking. Many employees are skeptical or unsure how to integrate AI tools into their workflows.

Why it’s important: In legacy companies, where resistance to change can be high, widespread adoption is critical for ensuring the AI delivers its intended value. Without it, even the best AI system can fall short.

How to approach it:

• Show the Results

  • Set up a dedicated, regular report highlighting milestones and successes.

• Engage Leadership Actively

  • Involve project sponsors in decision-making processes.

  • Organize monthly 'AI Strategy Sessions' where leaders interact with AI prototypes.

• Build an AI Learning Ecosystem

  • Develop a learning path from AI basics to advanced applications.

  • Launch an 'AI Certification Program' tailored to your organization.

• Cultivate an AI Champions Network

  • Identify and empower influential employees as AI advocates.

6. Showing Value

Scenario: Your AI project is live, but now you need to demonstrate its impact on the business.

Why it’s important: Proving the business value of AI is crucial for securing ongoing support and justify maintenance costs.

How to approach it:

• Establish Multilevel Performance Metrics

  • Define KPIs: Link AI performance to business outcomes like cost savings, efficiency gains, or customer satisfaction.

  • Create a Balanced Scorecard: Include technical, operational, and financial metrics.

  • Monitor Impact: Set up systems to track AI's effect on key business processes.

  • Develop a Value Dashboard: Integrate data from both legacy and AI systems for a comprehensive view.

• Collect Qualitative Feedback

  • Survey Users: Gather insights about their experience with the AI tool.

  • Highlight Testimonials: Document improvements and celebrate success stories.

• Leverage Comparative Analysis

  • Use A/B Testing: Showcase improvements by comparing AI-driven processes with previous methods.

  • Create Case Studies: Develop 'Before and After AI' examples to illustrate impact.

• Identify Internal Winners

  • Find Out Who 'Looks Good': Recognize individuals who benefit from the project's success—those seen as innovative or driving change.

  • Support Them: Help these stakeholders shine by providing data and stories they can share.

Conclusion

All of this is a lot to ask of those managing AI projects, but no one said it would be easy. That's why emphasizing the value of your AI initiatives is so important—not only to keep stakeholders motivated but also yourself. Scaling AI projects in legacy organizations is all about integrating AI into your organization's DNA to create sustainable value.

Next time, we'll explore continuous improvement and control mechanisms for AI projects.

Until then, stay innovative and keep pushing boundaries.

See you next Friday!

Tobias & Valentin