Nobel Prizes for AI

This InfoQ article from the cofounder of TitanML gives seven tips to those companies thinking about deploying a language model on-premises. The advantages of on-premise deployment include data privacy (since company data never leaves the company infrastructure), performance (especially when retrieval-augmented generation is linked to the model), and decreased costs in the long run. That said, self-hosting is a challenge because of the required computing resources. The author mentions that even a relatively small model, one with say seven billion parameters, still requires 14 GB of RAM and expensive GPUs. The seven tips are the following:

• Organizations should begin by being precise about production requirements. This includes clarifying whether they require real-time or batch latency for the model, the expected number of concurrent users, and the hardware available in the organization.
• The models deployed should always use quantization. This is a process to reduce memory and computing requirements by decreasing the bit width of model weights. For instance, this could involve reducing from 16-bit floating point to 8-bit integers.
• The organization should deploy GPU utilization strategies that optimize inference times in models. One approach is to batch inference processing. In dynamic batching, the team waits for a number of requests before treating them all, but this gives spiky performance. In continuous batching, multiple user requests are treated simultaneously by the model engine – in a manner analogous to an operating system that time-shares between a number of concurrently running applications. Parallelism is another technique to improve inference times. In this approach, processing is split over separate GPUs.
• The large computation requirements of AI models incentivize the centralization of all infrastructure resources.
• The deployment should be made future-proof. Since the lifetime of a model could be one year, until a more powerful model appears, organizations should ensure that the model can be migrated with minimum impact on clients and applications using the model.
• Though GPUs are expensive, they are still the best option for running generative AI models.
• Avoid large language models unless they are really needed. There is an increasing number of small models that are appropriate for many enterprise tasks.

Hugging Face has just launched Gradio 5 – a new version of an open source tool for creating machine learning Web applications. Gradio has over over 2 million monthly users and more than 470’000 applications built on the platform, and has been used in well-known solutions like Chatbot Arena, Open NotebookLM, and Stable Diffusion. The new release seeks to minimize the amount of Python (or JavaScript) code needed to create a model, and the AI Playground feature allows users to take any model available on Hugging Face and integrate it into their application. An independent security firm did a security audit of the platform and advised on the security measures that get integrated into Gradio 5 built applications. The platform will shortly include support for multipage apps, with native NAVs and sidebars, more media components and support for running applications on mobile devices.

This InfoQ article looks at several common prompt engineering techniques. Prompt engineering is the technique of phrasing prompts in a manner that yields better quality responses from the language model. The four prompt patterns reviewed are few-shot prompting, chain-of-thought prompting, self-consistency prompting, and tree-of-thought prompting. Prompt engineering is not (yet) a science, so experimentation and creativity are always encouraged in prompting.

Few-Shot Prompting. In AI, zero-shot prompting is when a language a language model is asked a question without being given any prior instructions. However, response quality is nearly always improved when some examples of the expected output are given in the prompt. This is few-shot prompting, for which the following prompt is an example.

The AI field has won Nobel prizes in chemistry and physics. In chemistry, the prize from the Royal Swedish Academy of Sciences was attributed to Google DeepMind CEO Demis Hassabis and John M. Jumper, who also works at DeepMind, for their work on using AI to predict protein structures. (The prize was also given to David Baker, a biochemistry professor at the University of Washington for work on computational protein design). Understanding a protein structure has always been a complex and long task, but the use of AI is offering new possibilities. Hassabis and Jumper used their AlphaFold AI to predict the structure of a protein from a sequence of amino acids in 2020. Since then, the tool has predicted the shapes of all currently known proteins. The ability to predict protein structures is critical to the development of efficient drugs, vaccines and even cures for cancer. DeepMind has released its latest model, AlphaFold 3, as open-source.

Geoffrey Hinton, one of the founding fathers of deep learning in the 1980s is a co-recipient of the 2024 Nobel Prize in physics with fellow computer scientist John Hopfield. The scientists developed neural network techniques by borrowing methods from physics. In recent years, Hinton, who also won the ACM Turing Award in 2018, and has been vocal in pointing to existential risks with AI, that include genocidal robots and economic collapse. He hopes that the Nobel prize will help bring more coverage of his arguments.

This article from the Guardian reports on a talk by the head of the Australian Service Intelligence Agency where he warned that social media is making it easier for people to get radicalized. He called social media “the world’s most potent incubator of extremism”, as it can take days or weeks to radicalize an individual, compared to months or years in previous generations. He cited the case of the Christchurch terrorist (a white supremacist who murdered 51 people in two mosques in 2019) who was radicalized via social media. The Intelligence chief specifically mentioned the Telegram App which has a chatroom called Terrorgram which is being used by individuals in Australia to communicate with terrorists abroad in order to plan terrorist attacks and to “provoke a race war”. He said that AI is being used by terrorists to improve their recruitment campaigns.

This article reports on a UN report that highlights how scammers in Southeast Asia are increasingly using AI in their pig butchering scams. In a pig butchering scam, the criminal builds an intimate relationship with the victim before asking for money, usually for an “investment opportunity”. An example use of AI is to overcome the language and dialect barriers that criminals have faced. Another is the creation of more convincing web-sites with job offers that scam potential job-seeking candidates, as well as fake commercial websites that contain crypto-draining malware. In this attack, the malware downloaded to the victim’s computer redirects crypto-currency payments to the criminals’ accounts. Another example cited from Western Africa are romance scammers that use real-time face replacement to masquerade as potential love interests. The UN suggests that criminals have already made 75 billion USD from pig butchering attacks. The money earned is enabling criminals to diversify their activities and to invest in more sophisticated technologies.

The US Department of Justice (DOJ) is proposing to address Google’s “illegal monopoly” in the online search and advertising space by splitting off the Chrome browser and Android operating system from the company. For the DOJ, Chrome browser’s default use of Google Search “significantly narrows the available channels of distribution and thus disincentivizes the emergence of new competition”. The DOJ said it seeks to “limit or end Google’s use of contracts, monopoly profits, and other tools to control or influence longstanding and emerging distribution channels and search-related products”. Android runs on 70% of the world’s smartphones and is the motor of Google’s advertising model with, the article says, 3 billion users worldwide generating trillions of search queries each year. The DOJ is also investigating agreements between Google and device makers that have led to Google being the default search engine on those devices. It is also looking at Google’s ability to collect vast amounts of user data, giving it a huge advantage over competitors for targeted advertising. Google might be obliged to share data with these competitors to level the playing field. In response, Google has argued that the high technical integration between Chrome and Android has allowed it to implement stronger security, and this could be jeopardized by any split.