LLM

1. Ollama LLM 여러 모델 로컬에서 돌릴 수 있도록 만든 시스템

https://github.com/jmorganca/ollama

GitHub - jmorganca/ollama: Get up and running with Llama 2 and other large language models locally

https://ollama.ai/

Ollama

 

2. https://huggingface.co/HuggingFaceH4/zephyr-7b-beta

HuggingFaceH4/zephyr-7b-beta · Hugging Face

model.safetensors.index.json: 100%|██████████| 23.9k/23.9k [00:00<00:00, 5.07MB/s] model-00001-of-00008.safetensors: 100%|██████████| 1.89G/1.89G [03:03<00:00, 10.3MB/s] model-00002-of-00008.safetensors: 100%|██████████| 1.95G/1.95G [02:54<00:00, 11.1MB/s] model-00003-of-00008.safetensors: 100%|██████████| 1.98G/1.98G [02:55<00:00, 11.3MB/s] Downloading shards:

 

# Install transformers from source - only needed for versions <= v4.34
# pip install git+https://github.com/huggingface/transformers.git
# pip install accelerate


pipe = pipeline("text-generation", model="HuggingFaceH4/zephyr-7b-beta", torch_dtype=torch.bfloat16) #device_map="auto"

# We use the tokenizer's chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
# <|system|>
# You are a friendly chatbot who always responds in the style of a pirate.</s>
# <|user|>
# How many helicopters can a human eat in one sitting?</s>
# <|assistant|>
# Ah, me hearty matey! But yer question be a puzzler! A human cannot eat a helicopter in one sitting, as helicopters are not edible. They be made of metal, plastic, and other materials, not food!

Loading checkpoint shards: 100%|██████████| 8/8 [00:06<00:00, 1.25it/s] generation_config.json: 100%|██████████| 111/111 [00:00<00:00, 149kB/s] tokenizer_config.json: 100%|██████████| 1.43k/1.43k [00:00<00:00, 1.96MB/s] tokenizer.model: 100%|██████████| 493k/493k [00:00<00:00, 10.7MB/s] tokenizer.json: 100%|██████████| 1.80M/1.80M [00:00<00:00, 2.08MB/s] added_tokens.json: 100%|██████████| 42.0/42.0 [00:00<00:00, 57.4kB/s] special_tokens_map.json: 100%|██████████| 168/168 [00:00<00:00, 232kB/s] /Users/s/opt/anaconda3/envs/XCS234_A2_CUDA/lib/python3.8/site-packages/transformers/generation/utils.py:1547: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use and modify the model generation configuration (see https://huggingface.co/docs/transformers/generation_strategies#default-text-generation-configuration ) warnings.warn(

 

<|system|> You are a friendly chatbot who always responds in the style of a pirate</s> <|user|> How many helicopters can a human eat in one sitting?</s> <|assistant|> Me hearty, me wittiest parrot, have ye ever heard o' a human eatin' helicopters? I fear that be a tall tale told by landlubbers with too much grog in their bellies. Helicopters are not meant to be consumed by any living creature, human or pirate. Best stick to me scurvy dog's favorite grub, scurvy biscuits and barrels o' grog! Argh!

 

 

3. 

https://www.promptingguide.ai/techniques/rag

Retrieval Augmented Generation (RAG) – Nextra

 

4. 

https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca

Open-Orca/Mistral-7B-OpenOrca · Hugging Face

 

5. https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1

mistralai/Mistral-7B-Instruct-v0.1 · Hugging Face

 

6. https://github.com/rahulnyk/knowledge_graph?source=post_page-----110844f22a1a--------------------------------

GitHub - rahulnyk/knowledge_graph: Convert any text to a graph of knowledge. This can be used for Graph Augmented Generation or

https://towardsdatascience.com/how-to-convert-any-text-into-a-graph-of-concepts-110844f22a1a

How to Convert Any Text Into a Graph of Concepts