Accelerate the development of Generative AI application with GitHub Models
The first step in developing generative AI applications is to choose a model. How to choose a model is the key. This includes
- When we combine application development with business scenarios, there are many comparisons, such as the generation effects of the same prompt words under different models.
- Quick comparison and switching of multiple models
- How different models adapt to new application frameworks and solutions to complete projects more effectively.
The release of GitHub Models plays a very important role for developers and different development teams to more effectively select models in the process of developing applications and create applications based on different application frameworks. Let’s take a look at how I use GitHub Models to complete development in different scenarios.
Model comparison
In GitHub Models, through the provided playground, we can complete the comparison of the same prompt for different models.
Let’s take a look at the comparison between Phi-3-mini and Mistral Nemo
Judging from the results, this is an evenly matched result.
Quick comparison and switching of multiple models
Above, we switched models in the playground to compare different models under the same prompt. For development, a more direct approach may be required. With the Azure AI Inference SDK you can quickly switch to different models. You can choose Python, JavaScript, and REST access methods by selecting Code.
If we choose the Phi-3-mini scenario, we can choose to obtain the access method in Code
Of course, you can directly and seamlessly access the programming environment through Codespace.
Adaptation to different application frameworks
Generative AI has different application frameworks combined with models to complete applications, such as GraphRAG. We can use the REST interface provided by GitHub Models to test model solutions other than GPT-4o, such as selecting the latest Meta LLama 3.1 405b Instruct. If the local deployment of this model has been limited by computing power, it will be difficult for individuals and small teams to adopt it. But based on the interface provided by GitHub Models, we can complete the test in the local environment very simply
- Configure the environment
Install the GraphRAG Python library
pip install graphrag -U
- Create a GraphRAG project
mkdir -p ./ragmd/input
python -m graphrag.index --init --root ./ragmd
- Modify settings.yaml
encoding_model: cl100k_base
skip_workflows: []
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_chat # or azure_openai_chat
model: meta-llama-3.1-405b-instruct
model_supports_json: true # recommended if this is available for your model.
max_tokens: 4000
api_base: https://models.inference.ai.azure.com
parallelization:
Stagger: 0.3
async_mode: threaded # or asyncio
embeddings:
async_mode: threaded # or asyncio
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_embedding # or azure_openai_embedding
model: jinaai
api_base: http://localhost:5146/v1
Note Please configure GitHub Tokens in .env
- Run
python -m graphrag.index --root ./ragmd
Test Results
python -m graphrag.query --root ./ragmd --method global "What's GraphRAG"
Through GitHub Models, we can quickly use the provided models for model comparison and application development environment testing, which allows model and application testing to be completed more efficiently and quickly in environments with limited computing power.
Learning Resources
-
Sign Up https://gh.io/models
-
Introducing GitHub Models: A new generation of AI engineers building on GitHub https://github.blog/news-insights/product-news/introducing-github-models/
-
Understand Phi-3 https://aka.ms/phi-3cookbook
-
Learn about GraphRAG https://microsoft.github.io/
Published on:
Learn more