Introduction to wx-tfjs-demo
The wx-tfjs-demo project serves as a demonstration of running TensorFlow within a WeChat mini-program. It showcases a practical integration of machine learning capabilities, specifically utilizing TensorFlow.js, to perform predictive analytics directly through the WeChat platform. This project regularly synchronizes its code with the WeChat mini-program known as "AI Pocket."
How to Run
Requirements
To successfully run the wx-tfjs-demo project, the following system and tool specifications are recommended:
- Operating System: MacOS
- NodeJS Version: v18.x.x
- Minimum WeChat Base Library Version: 2.29.0
- WeChat Developer Tools Version: v1.06.2210310 or higher
You should also configure the WeChat developer tools with the following settings:
- Enable "ES6 to ES5" transformation.
- Enable "Enhanced Compilation."
- For testing purposes, enable "Do not validate domain legality…”
Usage Instructions
The following steps should be followed to use the wx-tfjs-demo:
- Clone the project code to your local machine.
- Modify the
appidsetting in theproject.config.jsonfile. - Use
npm ito install dependencies (if necessary, usenpm i --force). - Compile dependencies using
npm run build. - Use a mobile device to scan the preview code generated by the WeChat developer tools.
Story Behind the Project
Early Implementation
Initially, the project required significant modifications to tfjs-core to enable TensorFlow.js to operate within a mini-program. A mini-program could use the camera to capture images displayed on a canvas. The mini-program's API allowed access to canvas data resembling ImageData, which could then be processed using TensorFlow.js for predictions.
For those interested in the challenges encountered during this stage, they are documented in blog posts such as trying tfjs on WeChat mini-program and retrying the TensorFlowJS porting.
Current Implementation
The current implementation leverages the elegant multi-platform support of TensorFlow.js, allowing the project to use a WeChat-specific plugin for essential functions like model loading, training, and prediction. As a result, the need for intrusive modifications to TensorFlow.js is eliminated.
Despite improved integration, challenges persist when it comes to ensuring accurate prediction results. The frames obtained through onCameraFrame differ from the displayed view, with variations even noted between the front and rear cameras on the same device. Nevertheless, a method for frame data cropping has been developed to help achieve better consistency across different platforms.
WeChat Mini-Program Demo
The project is part of the "AI Pocket" mini-program, a name chosen for its meaningfulness and significance. The project team is committed to developing the mini-program further, inviting users to experience it and provide feedback for improvements. Below is a QR code for "AI Pocket" to explore its features.
Collaboration and Communication
Collaboration Opportunities
The project's creator has extensive experience in full-stack development, Docker & Swarm, continuous deployment, and artificial intelligence, specifically in the NLP domain. This expertise enables them to quickly offer comprehensive solutions. Interested parties are encouraged to reach out for potential collaborations.
Moreover, the project is open source, and contributions from the community are welcome. While commercial use is permitted, users are urged to respect the effort invested in creating the project and to refrain from unethical practices. If the project proves beneficial, any form of appreciation or donation is welcome.
Communication Channels
Individuals can follow the creator's personal blog or the WeChat official account "Hunter's General Store" for regular technical insights and personal reflections. The QR code below provides access to the "AI Pocket" WeChat group—feel free to join for discussions.
