EmbeddingGemma 2: One 768-Dimension Space for Text, Images, Video and Audio. The Settings That Fail Silently

EmbeddingGemma 2 puts text, code, images, video and audio into one 768-dimension vector space, in an open model Google says is meant for laptops and phones. If you run retrieval or RAG, the practical question is what it costs to run and where it breaks.

This article uses Google's model card and developer guide. The benchmark numbers are Google's. I have not run the model.

What it is

Per the model card, EmbeddingGemma 2 is an open multimodal embedding model from Google DeepMind with 740M total parameters: a 270M text model (130M transformer plus 140M embedder), a 170M vision encoder and a 300M audio encoder. It builds on Gemma 4. The developer guide says the weights are released under the Apache 2.0 license.

Google lists these features:

• Four modalities (text including code, images, video, audio) in one 768-dimension space.
• 100+ languages, and a roughly 14% improvement on code tasks over the first EmbeddingGemma.
• An 8K-token context window shared by all modalities.
• Matryoshka Representation Learning, so vectors can be truncated to 512, 256 or 128 dimensions.
• Short text instruction prefixes that steer embeddings by task.

Load only what you need

The vision and audio encoders are separable. Google's table of effective sizes:

• Text only: 270M
• Text and image: 440M
• Text and audio: 570M
• Full multimodal: 740M

You disable encoders at load time with vision_config or audio_config set to None in config_kwargs. Google says disabled encoders are never loaded, so the savings apply to weights and peak memory. It also says the way to omit encoders differs between model libraries, so check the docs for yours.

Benchmarks, as Google reports them

At 768 dimensions, from the model card:

• MTEB multilingual v2: 61.36, versus 61.15 for EmbeddingGemma 1. That is a small gain on text.
• MTEB code v1: 78.68, versus 68.76.
• Image (MIEB lite): 64.64. MMEB v2 image: 57.28, visual documents: 67.84, video: 50.67.
• Audio retrieval (MSEB): 69.54. MAEB: 49.39.

The first version has no scores on the image, video and audio rows, so there is no like-for-like comparison there. The card does not compare against other vendors' models in the sections I read, so I cannot say how it ranks overall.

Truncation: cheap down to 256, not below

Google's truncation table (MTEB multilingual v2 and MMEB v2 overall):

Dimensions	Compression	MTEB multilingual	MMEB v2
768	1x	61.36	59.01
512	1.5x	61.17	58.38
256	3x	60.41	56.24
128	6x	57.89	45.65

Google says quality is close to lossless down to 256, and that 128 suits text-only workloads and should be validated on your own data for multimodal. The developer guide gives a storage example: a million 768-dimension vectors in bfloat16 take roughly 1.5 GB, and at 128 dimensions about 250 MB.

Settings that fail without an error

The model card is blunt about three mistakes that produce plausible output instead of a crash:

1. float16. Google says the activation range exceeds float16, so the model returns NaN or silently degraded embeddings. Use bfloat16 or float32. On most CPUs, float32.

2. No re-normalization after truncation. Slicing a unit-length vector breaks unit length. Google says skipping the L2 normalization degrades ranking quietly. Passing truncate_dim and normalize_embeddings=True to encode() handles it.

3. Mixed dimensions. A 768-dimension query cannot be scored against a 128-dimension corpus.

Also, the task prefixes are for text only. Google says omitting them still works but lowers precision. Documents with titles should be formatted as title: {title} | text: {content}, and prompt_name="Document" applies title: none, so titled documents have to be formatted by hand.

Context budget across modalities

All modalities share the 8,192-token window. Google's defaults: about 280 tokens per image (around 29 images), 140 tokens per video frame (around 58 frames), and 25 tokens per second of audio (around 327 seconds). Video is sampled at 1 frame per second by default, and audio should be 16 kHz mono. Google says a lower vision token budget can raise the limit to about 114 images or frames.

Where you can run it

The developer guide lists vLLM, Hugging Face Transformers, sentence-transformers, SGLang, MLX, Ollama, LMStudio and LiteRT, with weights on Hugging Face.

What I would check before switching

These are my own untested suggestions.

1. Re-embed a sample of your own corpus and compare retrieval against your current model. A 0.21-point gain on a text benchmark says little about your data.

2. Test at 256 dimensions first if storage matters, then 128 only if you are text-only.

3. Add a test that fails on NaN or non-unit-norm vectors, since both failures are silent.

4. Measure memory for the exact encoder combination you load.

What I would not conclude

• That it beats other embedding models. The sections I read do not compare against them.
• That the multimodal scores transfer to your content types.
• That 6x less storage is free. Google's own table shows the drop at 128 dimensions.

Frequently asked questions

Is EmbeddingGemma 2 open?

Google's developer guide says the weights are released under Apache 2.0.

How big is it?

740M parameters in full, 270M for text only, with 170M vision and 300M audio encoders loadable separately.

Which dimensions does it support?

768 natively, with Matryoshka truncation to 512, 256 and 128.

Can I use float16?

Google says no. It reports NaN or silently degraded output. Use bfloat16 or float32.

Do I need to normalize truncated vectors?

Yes, per Google. Re-normalize after truncating, or pass normalize_embeddings=True.

Does it need task prefixes?

They are recommended for text and improve precision. Images, video and audio take no prefix.

How long can inputs be?

8,192 tokens in total across all modalities in one input.

Sources

• Google AI for Developers, EmbeddingGemma 2 model card: architecture, benchmarks, truncation table, best practices, context limits.
• Google Developers Blog, EmbeddingGemma 2: The Developer Guide: license, tooling support, storage example.