OpenAI’s seven experiences in implementing enterprise AI: a practical guide from evaluation to automation

Recently, OpenAI released a 24-page collection of AI application cases, detailing how the world's top companies truly integrate artificial intelligence into their business processes and create real value. This report not only sparked widespread discussion in the industry, but also provided valuable reference for companies that are considering AI transformation.

In the report, OpenAI pointed out that AI has brought significant improvements in three key areas: improving employee performance , automating daily operations , and empowering product experience . Unlike traditional software deployment, successful AI implementation requires a new model, namely an "experiment-oriented" mindset and a "continuous iteration" working method.

Seven Experiences in Implementing AI in Enterprises

1. Start with Evals

Case: Morgan Stanley

As a global financial giant, Morgan Stanley verifies the performance of AI in specific scenarios through a rigorous evaluation process. They first focus on improving the efficiency of financial advisors through three evaluations: language translation accuracy, content summary quality, and comparison with expert advisor answers.

After systematic evaluation and continuous optimization, 98% of Morgan Stanley financial advisors now use AI tools every day, and the document access rate has jumped from 20% to 80%, greatly saving search time. Advisors can interact more deeply with clients, shortening follow-up communications that originally took several days to just a few hours.

The core of the assessment: Establish a systematic verification process around measurement indicators (accuracy, compliance, security, etc.) to ensure that AI output meets business requirements.

2. Integrate AI into the core of your product

Case: Indeed

Indeed, the world's largest recruitment website, has completely changed the job matching process using GPT-4o mini. They not only recommend jobs, but also use AI to generate personalized "recommendation reasons" to help job seekers understand why a certain job is suitable for them.

Through A/B testing, Indeed found that the new version of its matching engine based on GPT brought significant improvements: the number of job applications started increased by 20%, and the subsequent recruitment success rate increased by 13%. Considering that Indeed sends more than 20 million messages per month and has 350 million monthly website visits, these conversion rate improvements have had a huge impact on the business.

To improve efficiency, OpenAI worked with Indeed to fine-tune the small GPT model, maintaining similar results while reducing token usage by 60%.

3. Invest early to accelerate compounding effect

Example: Klarna

Klarna, a payment and shopping platform, has achieved comprehensive optimization of its customer service process through continuous investment in AI. Their AI assistant took on two-thirds of customer service conversations within a few months, equivalent to the workload of hundreds of customer service staff, shortening the problem resolution time from 11 minutes to 2 minutes, and is expected to bring the company a profit increase of US$40 million.

More importantly, 90% of Klarna’s employees currently use AI in their daily work, and the entire organization’s familiarity with AI continues to increase, enabling them to launch internal projects faster and continuously optimize the customer experience. This early investment has brought about the "compound growth" of AI benefits, continuously creating value in all aspects of the business.

4. Customize fine-tuning models to improve business accuracy

Example: Lowe's

Home improvement retail giant Lowe's has successfully improved the accuracy of product searches on its e-commerce platform by fine-tuning the OpenAI model. Faced with incomplete or inconsistent product data from thousands of suppliers, the fine-tuned model helped Lowe's improve product labeling accuracy by 20% and error detection capabilities by 60%.

Fine-tuning value: If the GPT model is a set of ready-made clothes, then fine-tuning is tailor-made. It can improve accuracy in specific areas, enhance industry expertise, maintain a unified brand style, and speed up workflows.

It is worth mentioning that OpenAI’s latest Vision Fine-Tuning function further enhances e-commerce search results and solves challenges in complex fields such as medical imaging and autonomous driving.

5. Put AI in the hands of experts to unleash innovation potential

Case: BBVA

Global banking giant BBVA has adopted an "employee-driven" approach to AI application. They have deployed ChatGPT Enterprise Edition globally, allowing employees to explore AI applications suitable for their own work scenarios.

In just five months, BBVA employees created more than 2,900 custom GPTs, some of which reduced project cycles from weeks to hours. The credit risk team uses AI to assess customer creditworthiness faster; the legal team uses it to answer approximately 40,000 policy compliance questions each year; and the customer service team automates sentiment analysis for NPS surveys.

BBVA’s secret to success is to hand over AI tools to those who understand the business best, allowing them to become the main innovators.

6. Remove obstacles for developers and build a unified platform

Case: Mercado Libre

Mercado Libre, the largest e-commerce company in Latin America, has created an AI development platform called Verdi, powered by GPT-4o and GPT-4o mini, to help 17,000 developers build AI applications faster and more uniformly.

Verdi integrates language models, Python nodes, and APIs into an extensible, unified platform with natural language as the core interaction method. Developers can quickly develop high-quality applications without having to delve into the underlying code, while security mechanisms and routing logic are built-in.

This platform has helped Mercado Libre achieve a number of breakthroughs:

• Product inventory building capacity increased 100 times

• Fraud detection accuracy close to 99%

• Automatically generate localized product descriptions

• Optimize product review summaries and personalized notifications

7. Set bold automation goals

Case: OpenAI’s own practice

OpenAI built an automation platform internally to help customer service teams handle repetitive tasks and speed up insight generation and action execution. The platform is integrated with Gmail, can access customer data and knowledge bases, automatically compose reply emails and trigger corresponding actions.

By integrating AI into existing processes, the OpenAI team is able to respond faster and focus more on customer needs. The platform handles hundreds of thousands of tasks per month, freeing up employees to do more impactful work, and is now expanding to other departments.

Common trends and future prospects

Successful AI implementation is not achieved overnight, but comes from an open and experimental mindset, supplemented by rigorous evaluation mechanisms and safety protection measures. Leading companies often focus on application scenarios with low investment and high returns, learn while doing, and then promote their experience to more fields.

OpenAI also introduced the latest Operator technology, which is an automation tool that can operate web pages, click buttons, and fill in forms like humans. Enterprises can use it to achieve end-to-end automation and directly update system data without programming instructions or APIs, thereby improving overall efficiency.

From these cases, we can see that AI has moved from concept to practice, and from experiment to large-scale application. For enterprises, what is important is not whether to adopt AI, but how to integrate AI into the core of business faster and more effectively to create sustainable competitive advantages.