Dolphin 3.0 represents the evolution of the Dolphin series, designed as a versatile and robust solution for various tasks such as coding, mathematical computations, agentic functions, and more. Unlike traditional models controlled by centralized platforms, Dolphin empowers users with complete autonomy. It allows businesses and individuals to define their own system prompts, tailor alignment to specific needs, and retain full control over their data. Dolphinâs unique approach ensures that it adapts to the userâs goals without imposing external restrictions or ethical guidelines. This model is built to serve as a flexible and dependable tool for innovation and problem-solving across diverse applications.
Model Resource
Hugging Face
Link:Â https://huggingface.co/cognitivecomputations/Dolphin3.0-Llama3.1-8B
Ollama
Link:Â https://ollama.com/library/dolphin3
Prerequisites for Installing and Running Dolphin 3.0 Llama 3.1 8B Locally
1. GPU:
- Memory (VRAM):
- Minimum: 12GB (with 8-bit or 4-bit quantization).
- Recommended: 24GB for smooth execution with FP16 or BF16 precision.
- Type: NVIDIA GPUs with Tensor Cores (e.g., RTX 3090, RTX A6000, A100, H100).
- Multi-GPU Setup (Optional for heavy workloads):
- 2x NVIDIA RTX 3090 or higher for handling larger batch sizes or faster inference.
2. Disk Space:
- Minimum: 25GB of free SSD space for storing model weights and temporary files.
- Recommended: 500GB SSD for storing additional checkpoints, logs, and datasets.
3. RAM:
- Minimum: 16GB RAM for basic model operations.
- Recommended: 32GB RAM for smoother execution, especially when preprocessing data or running multiple models.
4. CPU:
Recommended: 32 cores for optimal performance during preprocessing and parallelized tasks.
Minimum: 16 cores (multi-core processor).
Step-by-Step Process to Install Dolphin 3.0 Model Locally
For the purpose of this tutorial, we will use a GPU-powered Virtual Machine offered by NodeShift; however, you can replicate the same steps with any other cloud provider of your choice. NodeShift provides the most affordable Virtual Machines at a scale that meets GDPR, SOC2, and ISO27001 requirements.
Step 1: Sign Up and Set Up a NodeShift Cloud Account
Visit the NodeShift Platform and create an account. Once youâve signed up, log into your account.
Follow the account setup process and provide the necessary details and information.
Step 2: Create a GPU Node (Virtual Machine)
GPU Nodes are NodeShiftâs GPU Virtual Machines, on-demand resources equipped with diverse GPUs ranging from H100s to A100s. These GPU-powered VMs provide enhanced environmental control, allowing configuration adjustments for GPUs, CPUs, RAM, and Storage based on specific requirements.
Navigate to the menu on the left side. Select the GPU Nodes option, create a GPU Node in the Dashboard, click the Create GPU Node button, and create your first Virtual Machine deployment.
Step 3: Select a Model, Region, and Storage
In the âGPU Nodesâ tab, select a GPU Model and Storage according to your needs and the geographical region where you want to launch your model.
We will use 1x RTX A6000 GPU for this tutorial to achieve the fastest performance. However, you can choose a more affordable GPU with less VRAM if that better suits your requirements.
Step 4: Select Authentication Method
There are two authentication methods available: Password and SSH Key. SSH keys are a more secure option. To create them, please refer to our official documentation.
Step 5: Choose an Image
Next, you will need to choose an image for your Virtual Machine. We will deploy Dolphin 3.0 Llama 3.1 8B đŹ on an NVIDIA Cuda Virtual Machine. This proprietary, closed-source parallel computing platform will allow you to install Anything LLM on your GPU Node.
After choosing the image, click the âCreateâ button, and your Virtual Machine will be deployed.
Step 6: Virtual Machine Successfully Deployed
You will get visual confirmation that your node is up and running.
Step 7: Connect to GPUs using SSH
NodeShift GPUs can be connected to and controlled through a terminal using the SSH key provided during GPU creation.
Once your GPU Node deployment is successfully created and has reached the âRUNNINGâ status, you can navigate to the page of your GPU Deployment Instance. Then, click the âConnectâ button in the top right corner.
Now open your terminal and paste the proxy SSH IP or direct SSH IP.
Next, if you want to check the GPU details, run the command below:
nvidia-smi
Step 8: Install Ollama
After connecting to the terminal via SSH, itâs now time to install Ollama from the official Ollama website.
Website Link:Â https://ollama.com/
Run the following command to install the Ollama:
curl -fsSL https://ollama.com/install.sh | sh
Step 9: Serve Ollama
Run the following command to host the Ollama so that it can be accessed and utilized efficiently:
ollama serve
Now, âOllama is running.â
Step 10: Select Dolphin 3.0 Model
Link:Â https://ollama.com/library/dolphin3
Dolphin 3.0Â model is available in only one size: 70B. We will run it on our GPU virtual machine.
Step 11: Connect with SSH
Now, open a new tab in the terminal and reconnect using SSH.
Step 12: Check Commands
Run, the following command to see a list of available commands:
ollama
Step 13: Check Available Models
Run the following command to check if the downloaded model are available:
ollama list
Step 14: Pull Dolphin 3.0 Model
Run the following command to pull the Dolphin 3.0 model:
ollama pull dolphin3
Step 15: Run Dolphin 3.0 Model
Now, you can run the model in the terminal using the following command and interact with your model:
ollama run dolphin3
Note: This is a step-by-step guide for interacting with your model. It covers the first method for installing Dolphin 3.0 locally using Ollama and running it in the terminal.
Option 1: Using Ollama (Terminal)
- Install Ollama: Download and install the Ollama tool from the official site.
- Pull the Model:Â Run the following command to download the desired model:
ollama pull dolphin3
- Run the Model:Â Start the model in the terminal:
ollama run dolphin3
Option 2: Using Open WebUI
- Set Up Open WebUI:
Follow our Open WebUI Setup Guide to configure the interface. Ensure all dependencies are installed and the environment is correctly set up.
- Refresh the Interface:
Confirm that the Dolphin 3.0 model has been downloaded and is visible in the list of available models on the Open WebUI.
- Select Your Model:
Choose the Dolphin 3.0 model from the list. This model is available in a single size.
- Start Interaction:
Begin using the model by entering your queries in the interface. The Dolphin 3.0 is designed for high-quality instruction-based interactions, so input clear and detailed queries for the best results.
Conclusion
The Dolphin 3.0 Llama 3.1 8B model is a groundbreaking model from cognitivecomputations that offers advanced capabilities to developers and researchers. By following this step-by-step guide, you can easily install Dolphin 3.0 Llama 3.1 8B on a cloud-based virtual machine using a GPU-powered setup from NodeShift to maximize its potential. NodeShift provides a user-friendly, secure, and cost-effective platform to run your models efficiently. Itâs an ideal choice for those exploring Dolphin 3.0 Llama 3.1 8B and other cutting-edge models.
