Post

Little know gem the Open-source Cookbook

VLittle know gem: the Open-source Cookbook

A collection of notebooks for building practical AI applications using open-source tools and models: https://github.com/huggingface/cookbook

Doc: https://huggingface.co/learn/cookbook/index

Currently contains 16 notebooks in English and Chinese:

  1. Using LLM-as-a-judge ๐Ÿง‘โ€โš–๏ธ for an automated and versatile evaluation
  2. Create a legal preference dataset
  3. Suggestions for Data Annotation with SetFit in Zero-shot Text Classification
  4. Implementing semantic cache to improve a RAG system
  5. Building A RAG Ebook โ€œLibrarianโ€ Using LlamaIndex
  6. Stable Diffusion Interpolation
  7. Building A RAG System with Gemma, MongoDB and Open Source Models
  8. Prompt Tuning with PEFT Library
  9. Migrating from OpenAI to Open LLMs Using TGIโ€™s Messages API
  10. Automatic Embeddings with TEI through Inference Endpoints
  11. Simple RAG for GitHub issues using Hugging Face Zephyr and LangChain
  12. Embedding multimodal data for similarity search using ๐Ÿค— transformers, ๐Ÿค— datasets and FAISS
  13. Fine-tuning a Code LLM on Custom Code on a single GPU
  14. RAG Evaluation Using Synthetic data and LLM-As-A-Judge
  15. Advanced RAG on HuggingFace documentation using LangChain
  16. Detecting Issues in a Text Dataset with Cleanlab
Translate to Korean

๊ฑฐ์˜ ์•Œ๋ ค์ง€์ง€ ์•Š์€ ๋ณด์„ : ์˜คํ”ˆ ์†Œ์Šค ์š”๋ฆฌ ์ฑ…

์˜คํ”ˆ ์†Œ์Šค ๋„๊ตฌ ๋ฐ ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•˜์—ฌ ์‹ค์šฉ์ ์ธ AI ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์„ ๊ตฌ์ถ•ํ•˜๊ธฐ ์œ„ํ•œ ๋…ธํŠธ๋ถ ๋ชจ์Œ: https://github.com/huggingface/cookbook

๋ฌธ์„œ: https://huggingface.co/learn/cookbook/index

ํ˜„์žฌ ์˜์–ด์™€ ์ค‘๊ตญ์–ด๋กœ ๋œ 16๊ฐœ์˜ ๋…ธํŠธ๋ถ์ด ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค.

  1. ์ž๋™ํ™”๋œ ๋‹ค๋ชฉ์  ํ‰๊ฐ€๋ฅผ ์œ„ํ•ด LLM-as-a-judge ๐Ÿง‘ โš–๏ธ ์‚ฌ์šฉ
  2. ๋ฒ•์  ์„ ํ˜ธ๋„ ๋ฐ์ดํ„ฐ ์„ธํŠธ ๋งŒ๋“ค๊ธฐ
  3. Zero-shot ํ…์ŠคํŠธ ๋ถ„๋ฅ˜์—์„œ SetFit์„ ์‚ฌ์šฉํ•œ ๋ฐ์ดํ„ฐ ์ฃผ์„์— ๋Œ€ํ•œ ์ œ์•ˆ
  4. RAG ์‹œ์Šคํ…œ ๊ฐœ์„ ์„ ์œ„ํ•œ ์‹œ๋งจํ‹ฑ ์บ์‹œ ๊ตฌํ˜„
  5. LlamaIndex๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ RAG ์ „์ž์ฑ… โ€œLibrarianโ€ ๊ตฌ์ถ•
  6. ์•ˆ์ •๋˜์–ด ์žˆ๋Š” ์œ ํฌ ๋ณด์‚ฝ๋ฒ•
  7. Gemma, MongoDB ๋ฐ ์˜คํ”ˆ ์†Œ์Šค ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•œ RAG ์‹œ์Šคํ…œ ๊ตฌ์ถ•
  8. PEFT ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ์‚ฌ์šฉํ•œ ํ”„๋กฌํ”„ํŠธ ํŠœ๋‹
  9. TGI์˜ Messages API๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ OpenAI์—์„œ ๊ฐœ๋ฐฉํ˜• LLM์œผ๋กœ ๋งˆ์ด๊ทธ๋ ˆ์ด์…˜
  10. ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ํ†ตํ•œ TEI๋ฅผ ์‚ฌ์šฉํ•œ ์ž๋™ ์ž„๋ฒ ๋”ฉ
  11. Hugging Face Zephyr ๋ฐ LangChain์„ ์‚ฌ์šฉํ•œ GitHub ๋ฌธ์ œ์— ๋Œ€ํ•œ ๊ฐ„๋‹จํ•œ RAG
  12. ํŠธ๋žœ์Šคํฌ๋จธ, ๐Ÿค— ๋ฐ์ดํ„ฐ ์„ธํŠธ ๋ฐ FAISS๋ฅผ ์‚ฌ์šฉํ•œ ๐Ÿค— ์œ ์‚ฌ์„ฑ ๊ฒ€์ƒ‰์„ ์œ„ํ•œ ๋‹ค์ค‘ ๋ชจ๋“œ ๋ฐ์ดํ„ฐ ํฌํ•จ
  13. ๋‹จ์ผ GPU์˜ ์‚ฌ์šฉ์ž ์ง€์ • ์ฝ”๋“œ์—์„œ ์ฝ”๋“œ LLM ๋ฏธ์„ธ ์กฐ์ •
  14. ํ•ฉ์„ฑ ๋ฐ์ดํ„ฐ ๋ฐ LLM-As-A-Judge๋ฅผ ์‚ฌ์šฉํ•œ RAG ํ‰๊ฐ€
  15. LangChain์„ ์‚ฌ์šฉํ•œ HuggingFace ๋ฌธ์„œ์˜ ๊ณ ๊ธ‰ RAG
  16. Cleanlab์„ ์‚ฌ์šฉํ•˜์—ฌ ํ…์ŠคํŠธ ๋ฐ์ดํ„ฐ ์„ธํŠธ์—์„œ ๋ฌธ์ œ ๊ฐ์ง€
This post is licensed under CC BY 4.0 by the author.