How to run a LLM on your own computer.

If you’d like to explore how AI can enhance your business, reply to this email or contact us at [email protected] 

Before we get started with the main topic here is the latest news from this week:

The Latest News:

• Haters gonna hate. There has been a ton of bad press recently talking about how OpenAi is close to bankruptcy- burning through $700,000 a day. Not factoring in Microsoft’s $10 Billion investment in OpenAi, last week OpenAi announced that they are on track to make One Billion Dollars in revenue by the end of the year. This announcement was prior to the release of ChatGPT Enterprise. Haters gonna go figure?
  
  

• Super ChatGPT4. By itself GPT4 is incredibly powerful. But it is no small secret that by the use of clever prompting you can significantly improve its performance by up to 50%. AI Explained recently released a video detailing their prompting strategy which essentially boils down to:
  

  • Ideating multiple solutions to a problem.

  • Critiquing the multiple solutions.

  • Choose a solution based off the critiques.

  More useful however, Langchain quickly followed up Ai Explained’s video by updating their documentation with a tutorial on how you can implement this strategy. Even more useful I made a Google Colab Notebook so that you can try it yourself!
  

• Local LLMs Speculation Intensifies. A technique called “Speculative Sampling” made an appearance via a “Speculative PoC” pull request on the llama.cpp git repo. This technique improves the speed of local LLMs by 2x. Big enterprise are you paying attention to what's going on here?

Research Highlight:

This week’s highlight from our research is LM Studio. 

It is now easy to run a Local Large Language Model (LLM) on your computer.

Why would you want to do this?

• Privacy. If you are dealing with sensitive data, most clients would not be comfortable with their data traversing the internet to get to say chatGPT.

• Cost. If you are going to be making a repetitive set of calls to OpenAi - let's say doing data manipulation - the costs quickly add up. Running an LLM on your own computer is free.

• Internet Access. If you are in an area where you do not have internet, let's say an overseas flight or out in the countryside - it might still be super useful to have access to an LLM. By having one on your computer you still have access to specialised knowledge that only an LLM can provide!

Unfortunately until recently it has been very difficult to run a Large Language Model on your own computer. You had to have basic dev skills & understand a considerable amount on how the models themselves worked to even get started.

Recently however I discovered LM Studio and simply put, LM studio abstracts away all the pain.

Unlike all of the other solutions out there at the moment - LM Studio is simply an app.

All you need to do is visit their website & with a couple of clicks it is painlessly installed.

Best of all it is 100% free and works on both Windows & Mac.

You can get started yourself by visiting LM Studio’s website here

Note: we are not associated with LM Studio in anyway.