Turing is a prominent job platform dedicated to connecting pre-vetted remote software engineers with top U.S. firms within an impressive timeframe of 3-5 days. Leveraging AI, Turing manages a vast network of over 2 million developers worldwide and serves a diverse clientele of more than 900 customers.

As a data-driven company with a strong focus on AI, Turing utilizes an advanced AI-enabled platform to efficiently manage and index developers' information, including their skills, past experiences, geographical region, compensation, and more. By leveraging this comprehensive dataset, Turing ensures the best possible matches between developers and hiring companies. Staying on the cutting edge of AI technologies and rapidly implementing them into production is strategically vital for Turing.

The Challenges

The recent advancements in Large Language Model (LLM) technology have demonstrated its potential to revolutionize various industries. Many companies have actively started exploring the capabilities of GPT from OpenAI and other LLMs to enhance their products and internal operations. However, this exploration process comes with its own set of challenges.

The first challenge lies in the stark difference between the development environment for LLM features and the one used for existing products. Developing LLM features often necessitates the use of new libraries, which may rely on different versions of Python. Mixing them with existing product code bears an unnecessary risk of library conflict of breaking existing product. As LLM features usually start as independent components, it is best to have a sandboxed environment dedicated to their development. This challenge becomes even more pronounced when building internal tools that directly wrap around GPT-based toolkits such as AutoGPT and AgentGPT, which themselves rely on independent runtimes.

Asking individual engineers to set up their own development environments for LLMs not only adds to their workload but also creates management complications. Managing access keys, handling library updates, and introducing new toolchains are common issues that can slow down engineering efforts. Therefore, having an easy-to-manage and standardized development environment becomes crucial for maintaining engineering productivity in this domain.

Furthermore, engineers require efficient collaboration and rapid iteration when working on LLM features. They need a fast way to observe the new features in action, enabling teammates and stakeholders to interact with them and facilitate quick iterations. The standard release cycle for the core product, which often prioritizes stability over speed, falls short in meeting these needs. Kai Du, Director of Engineering at Turing, leading their LLM/GPT efforts commented:

We are in a new paradigm now, and we need a new process to stay competitive on this new frontier.

The Solution

The Turing team has effectively addressed these challenges and accelerated their development on LLMs by leveraging Crafting's on-cloud development environments. They specifically utilized the "Crafting for LLM" solution kit, which provides their engineers with an on-demand standard development environment that comes pre-installed with all the necessary tools.

The "Crafting for LLM" solution defines a Crafting Sandbox template, which includes a comprehensive toolchain for LLM feature development. On the lower level, it provides OpenAI key management, and is pre-installed with Python libraries for data processing and SDKs such as OpenAI SDK, langchain, etc.

In terms of dev tools, “Crafting for LLM” offers a familiar Jupyter Notebook interface for data scientists and WebIDE for engineers. Additionally, it provides a drag-and-drop file uploader for adding data files to the online workspace. Furthermore, the solution also sets up API servers using Python/Flask and Typescript/Nodejs, enabling engineers to quickly wrap their customized code in an API for integration with other services and UI, by adding just a few lines of code to the handler functions.

On top of that,  "Crafting for LLM" features an intuitive UI component that allows engineers to integrate their APIs into a chat-based user interface. It provides a shareable URL, enabling teammates to easily interact with the feature. The solution also includes built-in rating and commenting support, enabling collaborators to provide feedback and save it along with the conversation on the server for the engineer to review. This feedback loop enables rapid iteration of the feature.

Last but not least, the Crafting solution also integrates complete tools such as AutoGPT, AgentGPT in their own corresponding container for development, and allows the developer to add any tools easily either from source code checkout, or directly bringing in new containers. This greatly reduces the overhead of experimenting with new tools.

The Results

By utilizing Crafting's comprehensive solution, Turing has overcome the challenges associated with LLM development, providing their engineers with a powerful and efficient environment to accelerate their work.

Our experiments and developments on LLM have speeded up greatly with Crafting.

With the assistance of the Crafting solution, Turing engineers quickly built internal operation tools as well as experimented with new product features. One example of the solution's effectiveness is the swift development of a resume matching tool by Kai and his team. This tool enables the internal operations team to efficiently match developer roles posted by companies with available developer resumes using cutting-edge AI technology. Within just a few days since ideation, this powerful internal tool became readily accessible to operators, enhancing their productivity and workflow. Such remarkable speed gives Turing a significant advantage in the market.

Another notable example involves the quick experimentation with AgentGPT. By utilizing Crafting, developers are relieved of the burden of manual software installation and maintenance on their local machines. Additionally, as the Crafting environment is shared among team members, they can readily assist each other in configuring the tool and collaborate on finding the best scenarios to meet their needs. Even non-technical team members can swiftly create and utilize their own environments through the intuitive user interface provided by Crafting.

In summary, Crafting's comprehensive solution has greatly improved the iteration speed for Turing's LLM efforts.

We are very pleased to see the results and are actively looking to adopt Crafting for more use cases.