What is Mixpeek?

[Back to Blog](/blog)

[](https://twitter.com/intent/tweet?text=%F0%9F%8E%AF%20Multimodal%20Monday%20%233%20%E2%80%94%20Scaling%20Multimodal%20AI%3A%20Laws%2C%20Lightweights%20%26%20Large%20Releases&url=https%3A%2F%2Fmixpeek.com%2Fblog%2Fmultimodal-monday-3-scaling-multimodal-ai-laws-lightweights-large-releases)[](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fmixpeek.com%2Fblog%2Fmultimodal-monday-3-scaling-multimodal-ai-laws-lightweights-large-releases&title=%F0%9F%8E%AF%20Multimodal%20Monday%20%233%20%E2%80%94%20Scaling%20Multimodal%20AI%3A%20Laws%2C%20Lightweights%20%26%20Large%20Releases)[](https://www.reddit.com/submit?url=https%3A%2F%2Fmixpeek.com%2Fblog%2Fmultimodal-monday-3-scaling-multimodal-ai-laws-lightweights-large-releases&title=%F0%9F%8E%AF%20Multimodal%20Monday%20%233%20%E2%80%94%20Scaling%20Multimodal%20AI%3A%20Laws%2C%20Lightweights%20%26%20Large%20Releases)

This was originally in our newsletter.

Get Updates

Apr 14, 2025•6 min read

# 🎯 Multimodal Monday #3 — Scaling Multimodal AI: Laws, Lightweights & Large Releases

Apple’s new scaling law research redefines how multimodal models are built, while Moonshot and OpenGVLab drop powerful open-source VLMs with reasoning and tool-use.

![🎯 Multimodal Monday #3 — Scaling Multimodal AI: Laws, Lightweights & Large Releases](https://blog.mixpeek.com/content/images/2025/04/mm-3.png)

Multimodal Monday

TABLE OF CONTENTS

## 📢 Quick Take (TL;DR)

*   **Apple tests multimodal scaling:** A new _scaling laws_ study finds training vision+language models from scratch (early-fusion) can match or beat the usual “attach a vision encoder to an LLM” approach, hinting at simpler architectures ahead.
*   **Open-source VLMs on the rise:** Moonshot AI launched **Kimi-VL**, a lightweight 2.8B-parameter Vision-Language Model with reasoning and 128K context, while OpenGVLab’s **InternVL3** family (1B–78B params) pushes state-of-the-art multimodal reasoning and tool use.
*   **Real-world going multimodal:** Google’s Search AI can now “see” images via Lens+Gemini ([Google’s AI Mode can now see and search with images | The Verge](https://www.theverge.com/news/644363/google-search-ai-mode-multimodal-lens-image-recognition?ref=blog.mixpeek.com#:~:text=Google%20is%20adding%20multimodal%20capabilities,Mode%20to%20%E2%80%9Cmillions%20more%E2%80%9D%20users)), and industries from healthcare to retail are piloting AI that fuses modalities for better insights ([This Week in AI: Global Trends and Developments (Early April 2025)](https://www.linkedin.com/pulse/week-ai-global-trends-developments-early-april-2025-jitendra-kumar-7m4vf?ref=blog.mixpeek.com#:~:text=far%20fewer%20clinical%20deployments%20yet,smarter%2C%20more%20efficient%20healthcare%20system)

![AI Mode Lens gif](https://platform.theverge.com/wp-content/uploads/sites/2/2025/04/AI-Mode_Lens.gif?quality=90&strip=all&crop=0,0,100,100&_gl=1*142jsed*_ga*MTMyMDM2MjA3OS4xNzQ0NDg4MjI5*_ga_9GXHZT6RVE*MTc0NDQ4ODIyOC4xLjEuMTc0NDQ4ODIzMC41OC4wLjA.)

## 🧠 Research Highlights

*   **Scaling Laws for Native Multimodal Models (Apple):** Apple & Sorbonne researchers trained _457_ models to compare **early-fusion vs late-fusion** multimodal architectures. Key finding: no inherent advantage to “late-fusion” (e.g. a pre-trained vision encoder + LLM) over training a single multimodal model; in fact, early-fusion performs \*on par or slightly better at smaller scale. They also show these native models scale much like text-only LLMs and benefit from Mixture-of-Experts for modality-specific capabilities. _Why it matters:_ It challenges the prevalent strategy of bolting image encoders onto language models. Training one unified model from scratch might simplify multimodal AI development without sacrificing performance.
*   **Sparse Disentangled Retrieval Representations:** A new method from IIT Delhi/Adobe proposes **sparse, interpretable embeddings** for multimodal retrieval, enabling queries with exclusions (e.g. “sports **but not** basketball”). Instead of huge dense vectors, they map similar words/concepts into shared dimensions to keep embeddings compact. On MSCOCO and Conceptual Captions, this approach beat dense models like CLIP by up to **11%** in precision. _Why it matters:_ By making features more _interpretable_ and controllable, we can build search systems that answer complex queries (including negatives) and better explain **why** a result was retrieved – a big win for multimodal feature extraction transparency.

## 🛠️ Tools & Techniques

*   **Kimi-VL & Kimi-VL-Thinking (Moonshot AI):** An efficient open-source MoE vision-language model boasting advanced visual reasoning, long-context understanding, and agent capabilities, all with just ~2.8B active parameters([GitHub - MoonshotAI/Kimi-VL: Kimi-VL: Mixture-of-Experts Vision-Language Model for Multimodal Reasoning, Long-Context Understanding, and Strong Agent Capabilities](https://github.com/MoonshotAI/Kimi-VL?ref=blog.mixpeek.com#:~:text=We%20present%20Kimi,A3B)). It packs a _128K-token_ context window and a native-resolution Vision Transformer, enabling multi-image and document understanding.
    *   _How you might use it:_ The models (including the chain-of-thought tuned “Thinking” variant) are available on Hugging Face. Developers can run Kimi-VL on modest GPUs to power image Q&A, OCR with reasoning, or multi-turn visual assistants that would typically require much larger models.

![](https://github.com/MoonshotAI/Kimi-VL/raw/main/figures/arch.png)

*   **InternVL3 (OpenGVLab):** The **InternVL3** suite (from the team behind InternImage/InternLM) rolled out **7 checkpoint variants** from 1B up to 78B parame InternVL3 is out. It’s built with an InternViT vision encoder and Qwen2.5-VL language decoder, and it _outperforms the original Qwen-VL model_ by a good margin. These models demonstrate strong **visual reasoning**, can handle complex document understanding, and even exhibit _tool use and agentic capabilities_.

## 🏗️ Real-World Applications

*   **Google Search gets multimodal:** Google is **rolling out image understanding in Search’s AI Mode**. Users can now snap or upload a picture and ask questions about it, thanks to a custom Gemini AI model combined with Google Lens([Google’s AI Mode can now see and search with images | The Verge](https://www.theverge.com/news/644363/google-search-ai-mode-multimodal-lens-image-recognition?ref=blog.mixpeek.com#:~:text=Google%20is%20adding%20multimodal%20capabilities,Mode%20to%20%E2%80%9Cmillions%20more%E2%80%9D%20users)). The system can interpret an entire scene – recognizing objects, their relationships, and context – and then “fan out” search queries to give a rich answer (e.g. identify books in a photo and recommend similar ([Google’s AI Mode can now see and search with images | The Verge](https://www.theverge.com/news/644363/google-search-ai-mode-multimodal-lens-image-recognition?ref=blog.mixpeek.com#:~:text=%E2%80%9CAI%20Mode%20builds%20on%20our,%E2%80%9D)). This marks a major step in putting multimodal retrieval into a flagship consumer product.
*   **Multimodal AI in healthcare:** Healthcare is _on the cusp_ of mainstream adoption of multimodal AI. Hospitals are piloting diagnostic tools that analyze medical images alongside patient text data, and virtual health assistants that understand speech and vision ([This Week in AI: Global Trends and Developments (Early April 2025)](https://www.linkedin.com/pulse/week-ai-global-trends-developments-early-april-2025-jitendra-kumar-7m4vf?ref=blog.mixpeek.com#:~:text=so%20humans%20can%20focus%20on,now%20are%20laying%20the%20groundwork). Regulators are expected to green-light more medical AI systems soon, meaning tasks like radiology image analysis and personalized treatment recommendations could become routine. The goal: leverage combined modalities (e.g. MRI scans + clinical notes) to improve accuracy and assist doctors in care delivery ([This Week in AI: Global Trends and Developments (Early April 2025)](https://www.linkedin.com/pulse/week-ai-global-trends-developments-early-april-2025-jitendra-kumar-7m4vf?ref=blog.mixpeek.com#:~:text=far%20fewer%20clinical%20deployments%20yet,smarter%2C%20more%20efficient%20healthcare%20system).
*   **Industry use-cases expanding:** Beyond tech giants, sectors like **retail** are embracing multimodal feature extraction for better search and recommendation. E-commerce platforms now let shoppers search by image and text together for more precise results (think snapping a photo of a dress and adding the query “summer style”). According to market analysis, multimodal AI adoption is accelerating across retail, finance, and media, as organizations seek to fuse data types to gain richer insights ([Multimodal AI Research Report 2025: Market to Grow by Over](https://www.globenewswire.com/news-release/2025/04/08/3057833/0/en/Multimodal-AI-Research-Report-2025-Market-to-Grow-by-Over-25-Billion-by-2034-Opportunity-Growth-Drivers-Industry-Trend-Analysis-and-Forecasts.html?ref=blog.mixpeek.com#:~:text=Industry,financial%20services%20and%20streamline%20operations)). From automated video analysis in media archives to multimodal fraud detection in banking, the **real-world use cases** of multimodal systems are broadening by the week.

[![](https://blog.mixpeek.com/content/images/2025/04/image.png)](https://mixpeek.com/extractors?ref=blog.mixpeek.com)

[https://mixpeek.com/extractors](https://mixpeek.com/extractors?ref=blog.mixpeek.com)

## 📈 Trends & Predictions

*   **MOE for scalability:** _Mixture-of-Experts (MoE)_ is emerging as the go-to solution for scaling multimodal models. Meta’s Llama 4 uses MoE to juggle massive capacity—_Llama 4 Maverick_ reportedly has 128 experts (total 400B weights, 17B active) and achieves a **1 million-token** context window, while the “smaller” _Llama 4 Scout_ (109B total) hit an unprecedented **10 million-token** context ([Archive for April 2025](https://simonwillison.net/2025/Apr/?ref=blog.mixpeek.com#:~:text=Maverick%20is%20a%20400B%20model,token%20context%20length%E2%80%94an%20industry%20first)). We’re seeing others follow suit (e.g. Kimi-VL, InternVL3) by using experts to boost model size or speed. Expect future state-of-the-art VLMs to rely on MoE to attain huge contexts and improved efficiency simultaneously.
*   **Integrated reasoning:** There’s a clear pattern of multimodal models moving beyond perception into _cognition_. Several new models emphasize **chain-of-thought reasoning**: Google’s Gemini 2.5 “pauses to think” with an internal chain-of-thought approach ([This Week in AI: Global Trends and Developments (Early April 2025)](https://www.linkedin.com/pulse/week-ai-global-trends-developments-early-april-2025-jitendra-kumar-7m4vf?ref=blog.mixpeek.com#:~:text=Google%E2%80%99s%20AI%20division%20,tough%20math%20and%20science%20benchmarks), and Moonshot’s Kimi-VL-Thinking is explicitly trained for complex visual reasoning ([Kimi VL and Kimi VL Thinking: Powerful Open Source Vision Models](https://apidog.com/blog/kimi-vl-kimi-vl-thinking/?ref=blog.mixpeek.com#:~:text=Kimi%20VL%20Thinking%20takes%20this,it%20sees%E2%80%94it%20thinks%20about%20it)). Additionally, DeepMind CEO Demis Hassabis recently mentioned on the Possible podcast plans to merge Gemini with Veo's advanced video-generation models, aiming for multimodal assistants that deeply integrate visual understanding with reasoning to support real-world interactions. This trend suggests next-gen multimodal systems won’t just describe images or videos – they’ll perform logical reasoning on them (e.g. solve math problems from a diagram or plan tasks from an image). Multimodal AI is gradually shifting from “understand and tell” towards “understand and figure out,” blurring the line between perception and problem-solving.
*   **Keeping up with the flood:** The pace of multimodal AI progress isn’t slowing – if anything, it’s _accelerating_. Every week brings new model releases, whether it’s an open-source vision-language model or a specialized framework for cross-modal search. This rapid churn is giving rise to tooling to manage it. (For instance, **multimodal data warehouses** like [_Mixpeek_](https://mixpeek.com/?ref=blog.mixpeek.com) help teams by managing a library of feature extractors and retrieval pipelines so developers can plug in the latest and greatest models without reinventing the wheel.) A key prediction: organizations that invest in infrastructure to swiftly evaluate and swap in new multimodal models will stay ahead. Flexibility will be critical, because today’s state-of-the-art could be outpaced next month.

## 🧩 Community + Shoutouts

*   **Open-source heroes on X:** Shoutout to the researchers sharing their multimodal work openly. For example, @Kimi\_Moonshot unveiled _Kimi-VL_ on Twitter as a “lightweight yet powerful Vision-Language Model with reasoning capability" ([You Jiacheng on X: "MoonViT trained with Muon? Interesting ...](https://twitter.com/YouJiacheng/status/1910253172062577103?ref=blog.mixpeek.com#:~:text=You%20Jiacheng%20on%20X%3A%20,VLM%20and%20an%20MoE)), and @mervenoyann (OpenGVLab, now at Hugging Face) announced _InternVL3_ with sizes from 1B to 78B and support for reasoning.
*   **Viral _Ghibli-fication_ of images:** The public is having fun with multimodal AI too. Shortly after OpenAI’s **GPT-4o** (image-generation enabled GPT-4) went live, a _viral trend_ took off where users turned their personal photos into _Studio Ghibli–style art_. The craze was so popular it reportedly strained OpenAI’s ([This Week in AI: Global Trends and Developments (Early April 2025)](https://www.linkedin.com/pulse/week-ai-global-trends-developments-early-april-2025-jitendra-kumar-7m4vf?ref=blog.mixpeek.com#:~:text=load,voracious%20appetite%20for%20these%20tools)! It’s a lighthearted example of how quickly new multimodal capabilities get adopted by creators and casual users. From fake restaurant receipts to anime portraits, the community keeps finding creative (and unexpected) ways to use these tools.

![Ghibli Style Image: How to create Ghibli-style Portraits without paying for  ChatGPT Plus | - The Times of India](https://static.toiimg.com/thumb/msid-119581318,width-1280,height-720,imgsize-52248,resizemode-6,overlay-toi_sw,pt-32,y_pad-40/photo.jpg)

*   **Hugging Face hubs & demos:** Every week, new models and demos pop up on Hugging Face. Recent highlights include community Spaces that let you try out image-question answering with _open VLMs_ and visualization notebooks comparing embeddings from models like CLIP, BLIP-2, and the latest research prototypes. The Hugging Face community (shoutout to contributors like **@smol-vision** enthusiasts) are continuously evaluating and fine-tuning these models, which helps surface their strengths and weaknesses. If you’re not sure which new multimodal model works best for your task, chances are someone in the community has a benchmark or demo up for it already. 🎉

[PB

Philip Bankier

Philip’s expertise in RAG, data engineering, and AI agents makes him a natural fit to write about scaling multimodal data warehousing systems.



](/blog/authors/philip)

## Run this on your own content

Point Mixpeek at the storage you already have and search your video, images, audio, and documents the way this post describes. Build starts at $25/mo for up to 1M vectors.

[Search your own files](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=blog_onramp_end&utm_campaign=mvs_onramp)[Talk to us](/demo)

## Continue Learning

[

![Multimodal Monday #45: Birds, Whales, and the End of Latency](https://blog.mixpeek.com/content/images/2026/02/mm_45.png)

### Multimodal Monday #45: Birds, Whales, and the End of Latency

Your Weekly Multimodal AI Roundup (Feb 9 - Feb 16)

Read More





](/blog/multimodal-monday-45)[

![Multimodal Monday #44: Agents Ship Code, Robots Skateboard](https://blog.mixpeek.com/content/images/2026/02/mm44---with-milo.png)

### Multimodal Monday #44: Agents Ship Code, Robots Skateboard

Feb 2 - 9: GPT-5.3-Codex and Claude Opus 4.6 handle full software lifecycles from debug to deploy, MiniCPM-o 4.5 beats GPT-4o on vision tasks at 9B parameters running on-device, HUSKY skateboards using physics-based control, and TinyLoRA fine-tunes models with a single parameter.

Read More





](/blog/multimodal-monday-44)

### Author

PB

Philip Bankier

![Mixpeek Icon](/brand/logo-dark-icon.webp)

#### Multimodal Search, On Demand

Index, retrieve, and reason over videos, images, audio, and docs, all from a single API.

[Start Building](/start)

### Share

[](https://twitter.com/intent/tweet?text=%F0%9F%8E%AF%20Multimodal%20Monday%20%233%20%E2%80%94%20Scaling%20Multimodal%20AI%3A%20Laws%2C%20Lightweights%20%26%20Large%20Releases&url=https%3A%2F%2Fmixpeek.com%2Fblog%2Fmultimodal-monday-3-scaling-multimodal-ai-laws-lightweights-large-releases)[](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fmixpeek.com%2Fblog%2Fmultimodal-monday-3-scaling-multimodal-ai-laws-lightweights-large-releases&title=%F0%9F%8E%AF%20Multimodal%20Monday%20%233%20%E2%80%94%20Scaling%20Multimodal%20AI%3A%20Laws%2C%20Lightweights%20%26%20Large%20Releases)[](https://www.reddit.com/submit?url=https%3A%2F%2Fmixpeek.com%2Fblog%2Fmultimodal-monday-3-scaling-multimodal-ai-laws-lightweights-large-releases&title=%F0%9F%8E%AF%20Multimodal%20Monday%20%233%20%E2%80%94%20Scaling%20Multimodal%20AI%3A%20Laws%2C%20Lightweights%20%26%20Large%20Releases)