tech

Starting from January of this year, we have been holding a large model application landing salon in Shanghai every month, where I would like to share a few observations:

Regarding the development over the past year:

Cognitive alignment, blooming everywhere, some profitability, and anticipation for a hit product.

At the beginning of the year, some speeches at the meeting would still mention things like "our company has 5 years of AI experience," intentionally or unintentionally mixing LLM with previous AI technologies. Now, everyone can be very candid about which scenarios are using previous AI technologies and which are using models, and they can also clearly state what has been done on top of the models, and in what scenarios there have been fine-tuning or secondary training. This is a good phenomenon, allowing everyone to more directly enter the field that should be discussed.

Advertisement

At the same time, I have also seen more and more scenarios that have truly utilized LLM, including the combination of content generation with specific work scenarios, new RPA products that leverage LLM technology, and the real-world implementation in areas such as human resources interviews, screening, and job training.

For projects that have been implemented, some clients already have quantifiable data to evaluate the output of LLM, and there are also some service providers who have indicated that they have signed contracts or received project payments. Most of the projects undertaken by service providers seem to still be at the level of hundreds of thousands, with occasional projects in the low millions, and income from this direction has not yet become the main support for company revenue.

People were asking at the beginning of the year, and this question is still being asked now: what is the hit software for large models in the B2B sector? It seems there isn't one yet. Maybe there won't be for a long time? What have been the hit B2B software products over the years? Office automation? Financial ERP? CRM? DingTalk Feishu? Let's continue to look forward.

Regarding product form:

 Chat interface, traditional interface, traditional + chat = Copilot, and more?

For large model 2B software, there are various expectations, some pessimistic, but seemingly more are overly optimistic. We concretize these expectations by considering how everyone in a company will feel the impact of this new large model-driven enterprise software.

The large model first came into view with a chat interface. GPT-3.0 had already demonstrated capabilities very similar to the present, but because there was no chat, only an API, it did not attract more attention, nor did it break through to the mainstream. The chat interface and search may look similar, but once used, one can sense the fundamental differences. We have reason to believe that the next generation of enterprise information system infrastructure, with a chat interface that is accessible to everyone in the company, will become as ubiquitous in all enterprises as office software, email, and enterprise WeChat/DingTalk/Lark.

The startup partner incubated by Small is Big, colingo.ai, focuses on this market.

Traditional enterprise software has some backends that already use various AI algorithms. Replacing some of these with large models to solve problems better or using large models to handle what was previously unprocessable by algorithms has been happening quietly, but the perceptibility to end-users is relatively weak. These scenarios will become more common, but they may not easily produce hit software.

The combination of the chat interface and traditional GUI software is Copilot. As an early close partner of OpenAI, they launched Office Copilot, allowing users to better utilize traditional software through a chat interface. This also seems to be one of the main interface forms for a period of time?

What other new interface forms are there? In what form do digital employees cooperate with other employees? There is still much room for exploration in this area.

Regarding the technical route of 2B software service providers:

Top three approaches: orchestration, RAG (Retrieval-Augmented Generation), agentLower three paths: computing power, infrastructure, training and fine-tuning

Having more knowledge can actually be a burden, a phenomenon that indeed exists in the era of large models. Not to mention the concept of "large model thinking," which may be difficult to define, but it can be reflected in the process of various software service providers adopting large model applications.

The difference between the upper and lower three paths is not just a few GPUs; it involves talent, workflows, delivery cycles, iteration speeds, and overall comprehensive costs that are not on the same order of magnitude.

The first reaction of many companies' technical departments is to train models, buy cards, and recruit people. Then they may fall into a quagmire, and the feedback to the CEO becomes that large models are a heavy matter that requires significant investment. Internally, there is no progress, and externally, they have to face customers constantly asking what large models can do for them, leading to a dilemma.

The upper three paths can solve most of the problems involved in the application of large models. Companies should enter the lower three paths only after completing the upper three paths and confirming that the lower three paths can significantly improve results.

Banning the use of the word "training" in large model application development may be an overcorrection, but at this stage, for most application development companies, focusing on the upper three paths may be the fastest way to see results.

There are also two specific issues that I have discussed with many companies and would like to share with everyone:

Regarding which customers to start large model applications with:"Advanced enterprises use Feishu," we need to identify clients who have already embraced the "large model thinking." As a service provider, it is challenging to indoctrinate clients; instead, we should seek clients who are delighted by every improvement in results. If a client has a myriad of questions before even starting, let's steer clear of such users.

Regarding the choice of models:

Begin with the best model you have access to. If it's not feasible, then perhaps this generation of models is not yet capable; simply wait. Once it's done, use the most practical model for fine-tuning. Fine-tune to what extent, and also communicate this difference to the client. If they can accept it, proceed; if not, wait.

Do not consider the cost of the model. Today's prices are already incomparable to those from six months ago, and the capabilities six months from now will certainly surpass what we have today. When applying large models, have a basic belief in the "large model Moore's Law." Instead of being fixated on optimizing the current model, embrace time as your friend.

In summary, the judgment of opportunities is as follows:In the market for large model 2B applications, new scenarios, new forms, and new companies are emerging, while old scenarios are seeing new solutions and a mix of new and old players competing.

Undoubtedly, this is an opportunity for new enterprises, as the product forms are far from mature. Do you remember the mobile applications from the end of 2008, shortly after the iPhone was released in mid-2007? Similarly, the large models at the end of 2022 are just over a year and a half old. Of course, companies need to survive, but survival should not be the sole goal; there are many opportunities to explore, and if you don't seize them, someone else will.

Opportunities also belong to the current software service providers in various industries. 2B services ultimately need to solve users' problems, and you are the ones who understand user needs the best. In the application field, the competition in the era of large models has significantly reduced technical barriers. The ability to transform user needs into problems suitable for large model processing has become the most critical competitive point, which is undoubtedly an advantage for industry software service companies.

A reshuffling will inevitably occur, regardless of how long you have been entrenched in the industry. Whether you are the industry leader or the last in line, you could be eliminated in this competition, leaving a few spots for new companies native to large models, with the rest being reorganized in this technological revolution.

The decisive battle is this autumn, in the present year.

The first reordering will take place this autumn. Most companies did not have a specific budget for large models in 2023, with a few exceptions that diverted funds from other IT projects. In 2024, there will be some budgets allocated for proof-of-concept (POC) projects related to large models, and by 2025, many IT budgets will certainly be associated with large models. Whether you can make your solutions and your enterprise clients' perspectives stand out this autumn will determine how many projects you can secure in 2025 and your position in the first round of this ranking battle.