Shu Liu and Asim Biswal and Amog Kamsetty and Audrey Cheng and Luis Gaspar Schroeder and Liana Patel and Shiyi Cao and Simon Mo and Ion Stoica and Joseph Gonzalez and Matei Zaharia

EECS Department, University of California, Berkeley

Technical Report No. UCB/

May 1, 2025

Batch data analytics is a growing application for Large Language Models (LLMs). LLMs enable users to perform a wide range of natural language tasks, such as classification, entity extraction, and translation, over large datasets. However, LLM inference is highly costly and slow: for example, an NVIDIA L4 GPU running Llama3-8B can only process 6 KB of text per second, taking about a day to handle 15 GB of data; processing a similar amount of data costs around $10K on OpenAI’s GPT-4o. In this paper, we propose novel techniques that can significantly reduce the cost of LLM calls for relational data analytics workloads. Our key contribution is developing efficient algorithms for reordering the rows and the fields within each row of an input table to maximize key-value (KV) cache reuse when performing LLM serving. As such, our approach can be easily applied to existing analytics systems and serving platforms. Our evaluation shows that our solution can yield up to 3.4× improvement in job completion time on a benchmark of diverse LLM-based queries using Llama 3 models. Our solution also achieves a 32% cost savings under OpenAI and Anthropic pricing models.

Advisors: Ion Stoica

---

BibTeX citation:

@mastersthesis{Liu:31876,
    Author= {Liu, Shu and Biswal, Asim and Kamsetty, Amog and Cheng, Audrey and Schroeder, Luis Gaspar and Patel, Liana and Cao, Shiyi and Mo, Simon and Stoica, Ion and Gonzalez, Joseph and Zaharia, Matei},
    Title= {Optimizing LLM Queries in Relational Data Analytics Workloads},
    School= {EECS Department, University of California, Berkeley},
    Year= {2025},
    Number= {UCB/},
    Abstract= {Batch data analytics is a growing application for Large Language Models (LLMs). LLMs enable users to perform
a wide range of natural language tasks, such as classification, entity extraction, and translation, over large datasets.
However, LLM inference is highly costly and slow: for example, an NVIDIA L4 GPU running Llama3-8B can
only process 6 KB of text per second, taking about a day to handle 15 GB of data; processing a similar amount of
data costs around $10K on OpenAI’s GPT-4o. In this paper, we propose novel techniques that can significantly
reduce the cost of LLM calls for relational data analytics workloads. Our key contribution is developing efficient
algorithms for reordering the rows and the fields within each row of an input table to maximize key-value (KV)
cache reuse when performing LLM serving. As such, our approach can be easily applied to existing analytics
systems and serving platforms. Our evaluation shows that our solution can yield up to 3.4× improvement in job
completion time on a benchmark of diverse LLM-based queries using Llama 3 models. Our solution also achieves
a 32% cost savings under OpenAI and Anthropic pricing models.},
}

EndNote citation:

%0 Thesis
%A Liu, Shu 
%A Biswal, Asim 
%A Kamsetty, Amog 
%A Cheng, Audrey 
%A Schroeder, Luis Gaspar 
%A Patel, Liana 
%A Cao, Shiyi 
%A Mo, Simon 
%A Stoica, Ion 
%A Gonzalez, Joseph 
%A Zaharia, Matei 
%T Optimizing LLM Queries in Relational Data Analytics Workloads
%I EECS Department, University of California, Berkeley
%D 2025
%8 May 1
%@ UCB/
%F Liu:31876