Instructions to use trl-internal-testing/tiny-SmolVLMForConditionalGeneration with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use trl-internal-testing/tiny-SmolVLMForConditionalGeneration with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="trl-internal-testing/tiny-SmolVLMForConditionalGeneration") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("trl-internal-testing/tiny-SmolVLMForConditionalGeneration") model = AutoModelForImageTextToText.from_pretrained("trl-internal-testing/tiny-SmolVLMForConditionalGeneration") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use trl-internal-testing/tiny-SmolVLMForConditionalGeneration with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "trl-internal-testing/tiny-SmolVLMForConditionalGeneration" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "trl-internal-testing/tiny-SmolVLMForConditionalGeneration", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/trl-internal-testing/tiny-SmolVLMForConditionalGeneration
- SGLang
How to use trl-internal-testing/tiny-SmolVLMForConditionalGeneration with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "trl-internal-testing/tiny-SmolVLMForConditionalGeneration" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "trl-internal-testing/tiny-SmolVLMForConditionalGeneration", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "trl-internal-testing/tiny-SmolVLMForConditionalGeneration" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "trl-internal-testing/tiny-SmolVLMForConditionalGeneration", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use trl-internal-testing/tiny-SmolVLMForConditionalGeneration with Docker Model Runner:
docker model run hf.co/trl-internal-testing/tiny-SmolVLMForConditionalGeneration
| { | |
| "<end_of_utterance>": 49279, | |
| "<fake_token_around_image>": 49189, | |
| "<global-img>": 49152, | |
| "<image>": 49190, | |
| "<row_1_col_1>": 49153, | |
| "<row_1_col_2>": 49154, | |
| "<row_1_col_3>": 49155, | |
| "<row_1_col_4>": 49156, | |
| "<row_1_col_5>": 49157, | |
| "<row_1_col_6>": 49158, | |
| "<row_2_col_1>": 49159, | |
| "<row_2_col_2>": 49160, | |
| "<row_2_col_3>": 49161, | |
| "<row_2_col_4>": 49162, | |
| "<row_2_col_5>": 49163, | |
| "<row_2_col_6>": 49164, | |
| "<row_3_col_1>": 49165, | |
| "<row_3_col_2>": 49166, | |
| "<row_3_col_3>": 49167, | |
| "<row_3_col_4>": 49168, | |
| "<row_3_col_5>": 49169, | |
| "<row_3_col_6>": 49170, | |
| "<row_4_col_1>": 49171, | |
| "<row_4_col_2>": 49172, | |
| "<row_4_col_3>": 49173, | |
| "<row_4_col_4>": 49174, | |
| "<row_4_col_5>": 49175, | |
| "<row_4_col_6>": 49176, | |
| "<row_5_col_1>": 49177, | |
| "<row_5_col_2>": 49178, | |
| "<row_5_col_3>": 49179, | |
| "<row_5_col_4>": 49180, | |
| "<row_5_col_5>": 49181, | |
| "<row_5_col_6>": 49182, | |
| "<row_6_col_1>": 49183, | |
| "<row_6_col_2>": 49184, | |
| "<row_6_col_3>": 49185, | |
| "<row_6_col_4>": 49186, | |
| "<row_6_col_5>": 49187, | |
| "<row_6_col_6>": 49188, | |
| "<|reserved_special_token_0|>": 49191, | |
| "<|reserved_special_token_10|>": 49201, | |
| "<|reserved_special_token_11|>": 49202, | |
| "<|reserved_special_token_12|>": 49203, | |
| "<|reserved_special_token_13|>": 49204, | |
| "<|reserved_special_token_14|>": 49205, | |
| "<|reserved_special_token_15|>": 49206, | |
| "<|reserved_special_token_16|>": 49207, | |
| "<|reserved_special_token_17|>": 49208, | |
| "<|reserved_special_token_18|>": 49209, | |
| "<|reserved_special_token_19|>": 49210, | |
| "<|reserved_special_token_1|>": 49192, | |
| "<|reserved_special_token_20|>": 49211, | |
| "<|reserved_special_token_21|>": 49212, | |
| "<|reserved_special_token_22|>": 49213, | |
| "<|reserved_special_token_23|>": 49214, | |
| "<|reserved_special_token_24|>": 49215, | |
| "<|reserved_special_token_25|>": 49216, | |
| "<|reserved_special_token_26|>": 49217, | |
| "<|reserved_special_token_27|>": 49218, | |
| "<|reserved_special_token_28|>": 49219, | |
| "<|reserved_special_token_29|>": 49220, | |
| "<|reserved_special_token_2|>": 49193, | |
| "<|reserved_special_token_30|>": 49221, | |
| "<|reserved_special_token_31|>": 49222, | |
| "<|reserved_special_token_32|>": 49223, | |
| "<|reserved_special_token_33|>": 49224, | |
| "<|reserved_special_token_34|>": 49225, | |
| "<|reserved_special_token_35|>": 49226, | |
| "<|reserved_special_token_36|>": 49227, | |
| "<|reserved_special_token_37|>": 49228, | |
| "<|reserved_special_token_38|>": 49229, | |
| "<|reserved_special_token_39|>": 49230, | |
| "<|reserved_special_token_3|>": 49194, | |
| "<|reserved_special_token_40|>": 49231, | |
| "<|reserved_special_token_41|>": 49232, | |
| "<|reserved_special_token_42|>": 49233, | |
| "<|reserved_special_token_43|>": 49234, | |
| "<|reserved_special_token_44|>": 49235, | |
| "<|reserved_special_token_45|>": 49236, | |
| "<|reserved_special_token_46|>": 49237, | |
| "<|reserved_special_token_47|>": 49238, | |
| "<|reserved_special_token_48|>": 49239, | |
| "<|reserved_special_token_49|>": 49240, | |
| "<|reserved_special_token_4|>": 49195, | |
| "<|reserved_special_token_50|>": 49241, | |
| "<|reserved_special_token_51|>": 49242, | |
| "<|reserved_special_token_52|>": 49243, | |
| "<|reserved_special_token_53|>": 49244, | |
| "<|reserved_special_token_54|>": 49245, | |
| "<|reserved_special_token_55|>": 49246, | |
| "<|reserved_special_token_56|>": 49247, | |
| "<|reserved_special_token_57|>": 49248, | |
| "<|reserved_special_token_58|>": 49249, | |
| "<|reserved_special_token_59|>": 49250, | |
| "<|reserved_special_token_5|>": 49196, | |
| "<|reserved_special_token_60|>": 49251, | |
| "<|reserved_special_token_61|>": 49252, | |
| "<|reserved_special_token_62|>": 49253, | |
| "<|reserved_special_token_63|>": 49254, | |
| "<|reserved_special_token_64|>": 49255, | |
| "<|reserved_special_token_65|>": 49256, | |
| "<|reserved_special_token_66|>": 49257, | |
| "<|reserved_special_token_67|>": 49258, | |
| "<|reserved_special_token_68|>": 49259, | |
| "<|reserved_special_token_69|>": 49260, | |
| "<|reserved_special_token_6|>": 49197, | |
| "<|reserved_special_token_70|>": 49261, | |
| "<|reserved_special_token_71|>": 49262, | |
| "<|reserved_special_token_72|>": 49263, | |
| "<|reserved_special_token_73|>": 49264, | |
| "<|reserved_special_token_74|>": 49265, | |
| "<|reserved_special_token_75|>": 49266, | |
| "<|reserved_special_token_76|>": 49267, | |
| "<|reserved_special_token_77|>": 49268, | |
| "<|reserved_special_token_78|>": 49269, | |
| "<|reserved_special_token_79|>": 49270, | |
| "<|reserved_special_token_7|>": 49198, | |
| "<|reserved_special_token_80|>": 49271, | |
| "<|reserved_special_token_81|>": 49272, | |
| "<|reserved_special_token_82|>": 49273, | |
| "<|reserved_special_token_83|>": 49274, | |
| "<|reserved_special_token_84|>": 49275, | |
| "<|reserved_special_token_85|>": 49276, | |
| "<|reserved_special_token_86|>": 49277, | |
| "<|reserved_special_token_87|>": 49278, | |
| "<|reserved_special_token_8|>": 49199, | |
| "<|reserved_special_token_9|>": 49200 | |
| } | |