Technical concepts

Mixture of Experts (MoE)

Mixture of Experts (MoE) is a sparsely activated neural-network architecture. It combines several expert subnetworks with a gating network (router), activating only a few experts per inference step. This increases model capacity while controlling computation, separating capacity from compute cost. Mixtral helped popularize open-source MoE, and current open-source flagships such as DeepSeek V4 and Llama 4 commonly use this architecture.

Vision Transformer (ViT)

A Vision Transformer (ViT) divides an image into patches, treating each patch as a token for a Transformer encoder. This challenged the dominance of convolutional networks in computer vision. Work such as CLIP and LLaVA connected image and text semantics on this foundation. Most visual encoders in current multimodal models follow ViT principles. Important variants include DeiT, Swin Transformer, BEiT and MAE.

Multimodal models

Multimodal models process and understand several data types, including text, images, audio and video. There are three main approaches: encoder-decoder architectures that attach a visual encoder to a language model; contrastive learning such as CLIP's image-text alignment; and native multimodal architectures that mix modalities during pre-training. Current flagships such as gpt-5.6-sol, claude-opus-4-8, claude-sonnet-5 support multimodality. Typical applications include image understanding and descriptions, visual question answering, document and table parsing, chart interpretation and creative generation.

Diffusion models

Diffusion models learn data distributions through forward noising and reverse denoising. Training gradually adds noise to samples; inference starts from pure noise and progressively reconstructs high-quality images, audio or video. Stable Diffusion, Midjourney and the Veo video family rely on diffusion processes or related variants. DDPM established the theoretical foundation; Latent Diffusion moved denoising into latent space to reduce cost; DiT replaced U-Net with Transformers as a mainstream backbone. Common control techniques include ControlNet, IP-Adapter and LoRA fine-tuning.

Prompt optimization

A prompt is the key input used to interact with a large language model. Improving the prompt often improves output quality more directly than changing models.

• Basic principles: Give clear instructions, sufficient context and an explicit output format.
• Suggested structure: Role + task + input + output requirements + examples + considerations.
• Advanced techniques: Few-shot examples, chain of thought, role prompting, decomposing complex tasks and requesting self-verification.
• Common mistakes: Ambiguous wording, overly open-ended tasks and reliance on negative instructions. Prefer positive instructions describing the desired behavior.