Shocking news! WeClone opens a new era of digital immortality, creating your own digital clone

WeClone: ​​Create your own digital avatar

With the rapid development of artificial intelligence, personalized digital avatars have gradually become a reality, from a science fiction concept. Today, we will introduce to you the highly anticipated open source project on GitHub - WeClone, which can use deep learning technology to integrate WeChat chat records and voice data to create a unique AI digital avatar for you.

1. Project Overview

WeClone aims to train a highly personalized dialogue model through the user's WeChat chat history, realize the "digital version of you", and explore the possibility of "digital immortality" to a certain extent. It innovatively integrates large language models and speech synthesis technology to provide a digital cloning solution for the WeChat ecosystem. The system trains a personalized language model by analyzing the user's historical chat records, and can also use a 0.5B parameter large model to process WeChat voice messages to generate cloned voices with a similarity of 95% to the original voiceprint.

GitHub address: https://github.com/xming521/weclone

2. Core Functions

1. Chat record-driven personality model training

1. Data collection and preprocessing : Supports convenient export of WeChat chat records and automatically processes them into question and answer format. The project removes sensitive information such as mobile phone number, ID number, email address, website, etc. from the data by default. It also provides a dictionary of banned words, and you can add filter words by yourself. At the same time, there are multiple ways to deal with the situation where the same person answers multiple sentences in a row. For example, use PyWxDump to extract WeChat chat records. After downloading the software and decrypting the database, click Chat Backup, export type to CSV, and export the exported files inwxdump_tmp/exportofcsvFolder in./datasetDirectory.

2. Model fine-tuning : Based on the LoRA method, low-resource fine-tuning is performed on mainstream 0.5B - 7B scale models, such as ChatGLM3 - 6B, Qwen2.5 - 7B, etc. Using ChatGLM3 - 6B as the base model, fine-tuning is performed in the SFT stage, using a low-rank adapter, significantly reducing trainable parameters, supporting single-machine/distributed training, and compatible with multi-card training acceleration. Using the Qwen2.5 - 7B - Instruct model for LoRA fine-tuning requires approximately 16GB of video memory. Users can choose the appropriate model and training method based on hardware conditions and data volume.

   
   

2. High-fidelity voiceprint cloning system (WeClone - audio)

As a supporting submodule of the project, based on the lightweight Tacotron or WavLM model, using a voice cloning network with a parameter size of about 0.5B, using a 5-second voice sample, it is possible to clone a voice with a similarity of up to 95%. For example, using the Spark - TTS solution, only 4GB of video memory is required, and detailed voice control such as pitch and speech speed is also supported, further enhancing the realism of the digital avatar.

3. Multi-platform chatbot real-time interaction framework

Through the AstrBot framework, you can deploy your digital avatar to multiple chat platforms such as WeChat, QQ, Telegram, WeChat for Business, and Lark. You can quickly start a real-time conversation with your digital avatar with just one line of command. For example, deploy a messaging platform in AstrBot and executepython weclone/server/``api_service.pyStart the API service, add a new service provider in AstrBot, select OpenAI as the type, fill in the API Base URL according to the AstrBot deployment method, fill in gpt-3.5-turbo as the model, and fill in any API Key to complete the initial deployment.

3. Application Scenarios

1. Personal assistant customization

When you are busy, your digital twin can reply to messages and handle daily affairs on your behalf, such as writing emails and replying to comments. For example, when you are in a meeting and have no time to take care of your phone, your digital twin can automatically reply to WeChat messages based on your language style, so that you can maintain normal communication with friends and family.

2. Content creation assistance

Quickly produce personalized text content in a specific style. Users who operate self-media can use digital avatars to write tweets, scripts, commentaries, etc. to keep the style of multiple accounts consistent. For example, a food blogger uses his chat records to train his digital avatar to help create food recommendation copywriting, which not only saves time, but also ensures that the copywriting style is close to the blogger himself, which is more popular with fans.

3. Digital Eternal Commemoration

Create a digital avatar of yourself or others to preserve precious memories. For those who have lost their loved ones or friends, creating a digital avatar through their WeChat chat records makes it seem as if the other person is still by their side, achieving "digital immortality" to a certain extent and comforting the soul.

4. Technical Architecture Analysis

1. LLM-based dialogue fine-tuning module

1. Model selection : ChatGLM3-6B is selected as the basic model, which has strong language understanding and generation capabilities, providing a solid foundation for personalized fine-tuning.

2. Fine-tuning technology : Using the LoRA method, while maintaining the basic capabilities of the model, a personalized conversation style is injected by adjusting a small number of parameters. This technology significantly reduces the demand for video memory, allowing model training to be completed efficiently on ordinary hardware.

2. WeClone - audio voice cloning module

1. Solution 1: Spark - TTS : Recommended solution, with low resource requirements, 0.5B model only needs 4GB of video memory. It supports WeChat voice messages as input, and the maximum supported voice segment is 15 seconds. It can achieve accurate conversion from text to speech, maintain the stability of sound features, and support detailed voice parameter adjustment, such as pitch, speaking speed, etc.

2. Option 2: Llasa : supports two specifications: 1B (9GB video memory) and 3B (16GB video memory). It can also achieve high-quality sound cloning, providing options for users with different hardware conditions.

5. Usage Guide

1. Environment Construction

It is recommended to use uv (a fast Python environment manager) to create a new Python environment and install dependencies (note that the dependencies for the audio cloning function are not included). After installing uv, use the corresponding commands to create an environment and install dependencies.

2. Data preparation

Use PyWxDump to extract WeChat chat records, export them to CSV format, and store the exported files in the corresponding directory according to the specified structure.

3. Data preprocessing

Run the WeClone providedweclone - cli make - datasetThe command cleans the extracted CSV file, and you can set keywords to filter sensitive information. You can also use the large language model to score the chat data and filter out conversations with low matching degrees.

4. Model training

Adjustmentsettings.jsoncThe training parameters in the file are runweclone-cli train-sftStart training. After the training is completed, the generated LoRA file will appear in the specified directory.

5. Model testing and deployment

You can test common chat problems through the browser demo or interface reasoning function. Deploy the trained model to the chat platform supported by the AstrBot framework to achieve real-time interaction with the digital avatar.

6. Project Advantages

1. Low threshold

The hardware requirements are moderate, and an ordinary computer equipped with 16GB video memory can run it. At the same time, the project adopts a modular design, and each functional module can be used independently, which reduces the difficulty of development and facilitates developers to carry out secondary development and customization according to their own needs.

2. High Customizability

Supports personalized conversation style training. By fine-tuning the model, the digital avatar can accurately reproduce the user's expression habits, language style and even catchphrases. Customizable filtering vocabulary, flexible control of training data, and data security and personalized needs are guaranteed.

(III) Safety considerations

Built-in privacy information filtering mechanism automatically removes sensitive information. The project also supports local operation, and data is stored on local devices to maximize user data security.

VII. Precautions for use

1. Data quality requirements

The recommended number of chat records is more than 20,000, and the recommended voice sample is within 15 seconds. Ensure the quality and diversity of training data to obtain a more ideal digital avatar effect. If the chat records are sparse or inconsistent in style, the generated results may be unstable.

2. Windows compatibility

The project is mainly developed for the Linux platform. Windows users are recommended to use WSL (Windows Subsystem for Linux) to ensure the stable operation of the project.

3. Version iteration

The current project is in a rapid development stage. Some functions may be unstable or have interface changes. When using it, you need to pay attention to the official updates of the project and adjust the usage in time.

4. Privacy and Compliance

Chat data involves a large amount of personal sensitive information. During use, the Personal Information Protection Law and other relevant laws and regulations must be strictly observed, and digital avatars must not be used for illegal purposes.

8. Future Outlook

The WeClone project is still developing. In the future, we plan to add RAG (Retrieval Enhanced Generation) technology to support access to more knowledge bases, optimize knowledge retrieval efficiency, and improve the knowledge reserve and answer accuracy of the digital twin. At the same time, we will increase multimodal support to achieve image understanding and generation, video content processing, and cross-modal interaction capabilities, making the digital twin more powerful and more natural.

In short, WeClone opens the door to personalized AI digital avatars for us, showing great potential and application value in both personal life and work scenarios. Interested friends may wish to go to the project GitHub page to learn more and try it out, and explore the wonderful world of digital avatars together.