A few days ago, OpenAI released Sora, a video generation model, which aroused widespread concern and discussion around the world. From the emergence of deep learning to the wave of big models sweeping the science and technology circle, the development of AI has experienced many milestone events. Among them, the Transformer architecture can be said to be the "ruler" in the field of large models. What role does the Transformer architecture play for the AI big model? How to deal with the challenges of computing, storage, interconnection and other aspects of computing infrastructure supporting large models? What kind of big model and new architecture is likely to be a substitute for Transformer? Recently, in view of the hot issues concerned by the industry, the reporter of People’s Posts and Telecommunications interviewed Professor Xiao Limin, director of the computer department of Beihang University.
Four advantages make Transformer architecture sweep the AI field.
Since it was put forward in 2017, Transformer has surpassed the recurrent neural network (RNN) model with its excellent language understanding ability, and quickly emerged in the field of natural language processing (NLP), and later entered the field of images, breaking the dominance of convolutional neural network (CNN). Transformer’s powerful presentation ability, parallel computing ability and cross-domain application ability have laid the foundation for the breakthrough of AI big model.
Xiao Limin explained that the Transformer architecture is a neural network model based on self-attention mechanism, which solves the problems of computational efficiency and difficulty in parallelization of traditional RNN models when dealing with long sequence data. He further elaborated the four main functions and advantages of the Transformer architecture in the AI big model.
First, Transformer captures the global dependence in the input sequence through the self-attention mechanism, which solves the problems existing in the traditional neural network model when dealing with long sequence data. This mechanism can make the model dynamically pay attention to different parts of the input sequence, and improve the model’s ability to accurately process contextual information.
Secondly, Transformer improves the training efficiency and stability of the model by introducing techniques such as residual connection and layer normalization. Compared with the traditional RNN and CNN models, Transformer is more efficient and stable when dealing with long sequence data.
Third, the self-attention and multi-attention mechanism, hierarchical structure and scalability of Transformer architecture make it an easy-to-scale and parallel model, which is very suitable for large-scale distributed computing.
Fourthly, Transformer architecture has been widely used in the field of natural language processing, such as machine translation, text classification and generation. At the same time, with the in-depth research and improvement of Transformer architecture, it is gradually applied to image processing, audio processing, recommendation system and other fields.
The challenge brought by the big model "violence aesthetics" deserves attention.
Computing infrastructure is the computing base supporting the large model. With the blessing of large computing power, when the amount of training data, the scale of model parameters and the degree of training adequacy reach a certain level, some capabilities of the large model in certain types of tasks can "emerge intelligently", which is the "violent aesthetics" of the AI large model. The parameter scale of large-scale model is hundreds of billions or even trillions, and the complexity of large-scale distributed training is high, which puts forward new demands and challenges for computing power infrastructure, such as large computing power, large memory, high access bandwidth and high-speed interconnection.
Xiao Limin said that we should pay attention to the efficient coordination of multi-level software and hardware in computing infrastructure. By analyzing the adaptation between the load characteristics of large model and the structure of intelligent computing system, the strategy of load segmentation and parallelization of large model is optimized, and the efficient training and reasoning of large model are realized through the multi-level vertical cooperation of model, framework and intelligent computing system and the horizontal cooperation of software and hardware resources such as computing, storage and interconnection. In addition, it is also a beneficial attempt to realize hybrid computing by using various types of computing chips. By aggregating different types of computing chip resources, a multi-heterogeneous computing resource pool is formed, which is used to support the same model training task through heterogeneous programming, real-time compilation, resource awareness, load splitting and dynamic scheduling, so as to maximize the utilization of various computing resources.
At the same time, Xiao Limin believes that technical approaches to improve the efficiency of large model training and reasoning can also be explored from all key links of computing infrastructure such as computing, storage, interconnection and software stack.
First, in terms of computing, the development or selection of computing power chips is very important. We should not only consider the current situation that the mainstream architecture of large models tends to focus on Transformer, but also consider the possible evolution of large model architecture in the future. The "universal+customized" chip architecture can be regarded as an idea of having both, for example, customizing and accelerating the Transformer in the general GPU architecture, taking into account the needs of large models for chip adaptability and energy efficiency. In addition, we can also explore new chip architectures such as "integration of storage and calculation", and integrate computing and storage units into the same chip, thus greatly reducing the delay and energy consumption of data transportation between the processor and the memory.
Second, in terms of storage, HBM technology can provide large-capacity storage and high-bandwidth memory access capability, or CXL technology can realize large memory pool sharing, which can better carry massive data such as training samples, weight parameters and optimizer status of large models, and improve memory access performance and storage utilization efficiency in large model training scenarios.
Thirdly, in terms of interconnection, the data interaction overhead of large-scale model training can be reduced through high-speed interconnection mechanism between chips such as NVLink and HCCS, RDMA direct interconnection between GPU nodes, accelerated data exchange between video memory and GPU Direct RDMA, and optimization of aggregate communication functions such as AllReduce and AlltoAll.
Fourth, in the software stack, we should focus on breaking through such technologies as automatic segmentation and parallelization tools for large model loads, elastic fault-tolerant mechanism for distributed training of large models, operator library for optimal adaptation of model loads and chip structure, unified management and efficient scheduling system for multi-heterogeneous computing power, cross-chip architecture and adaptive optimization programming language and compilation tool chain, and establish a full-stack software environment to bridge the upper-level large model application and the lower-level hardware resources efficiently.
Exploring the Optimization Way of Transformer Architecture
At present, almost all AI big models are stacked on the basis of Transformer architecture. From BERT, GPT series, Flan-T5 and other language models to SAM, Stable Diffusion and other image models, Transformer is sweeping the world of AI big models with an irresistible trend.
Of course, Transformer also has some inherent shortcomings. Xiao Limin explained that the current AI model has been able to handle longer sequences and higher-dimensional representations, thus supporting longer contexts and higher quality. However, the time and spatial complexity of the current Transformer architecture shows a quadratic growth pattern in sequence length and/or model dimensions, which will not only limit the context length, but also increase the expansion cost. In order to solve these problems, people are constantly exploring ways to optimize the algorithm and improve the network structure, in order to improve the performance of Transformer and break its limitations.
Recently, the research teams of Stanford University and the State University of New York at Buffalo proposed a new alternative technology: Monarch Mixer, which abandoned the high-cost attention and MLP mechanism in Transformer and replaced it with the expressive Monarch matrix with secondary computational complexity. In language and image experiments, Monarch Mixer achieved better performance with lower computational cost.
Earlier, Chris Re Laboratory of Stanford University put forward a series of sub-quadratic alternative architectures in its research named S4, which can be comparable to the performance of Transformer architecture in specific situations and can significantly reduce the amount of computation.
Xiao Limin said that the big model based on Transformer still has a well-known deficiency-the bias problem caused by the unbalanced training set. To solve this problem, another research team of Stanford University proposed a new neural network architecture, Backpack, which intervened the behavior of the language model by regulating the meaning vector, thus guiding the language model to output the expected results.
In addition, several other research teams have proposed a number of Transformer alternative architectures for the purpose of improving computational efficiency, such as Microsoft’s RetNet, UniRepLKNet jointly released by Tencent and the Chinese University of Hong Kong, and StripedHyena of OgetherAI.
Xiao Limin believes that some alternative architectures of Transformer that people are currently exploring will bring multiple potential benefits to the development of AI big model. The first is the reduction of computational complexity, which makes the parameter scale of the model smaller under the same target performance, and then reduces the demand for computational power. Or the performance of the model can be better with the support of the same parameter scale and computing power.
It is worth mentioning that if the low-parameter and high-performance characteristics of the alternative architecture reach a certain level, the deployment threshold of the large model on low-computing equipment can also be lowered, thus further expanding the application scope and scenarios of the large model. In addition, some alternative architectures can effectively eliminate the problem of model bias, make the model more controllable and interpretable, and enable users to guide the model to output the desired results through appropriate intervention.
Although there are many potential alternative architectures of Transformer at present, there will definitely be more competitors or eventually substitutes in the future, it is expected that Transformer will remain the mainstream architecture in the field of large models for a long time.
