Machine Translation Models for Structured Query Construction from Natural Language Queries: A Case Study

Abstract

This study presents a comparison of structured query construction methods from natural language queries for custom proprietary data. We implemented three machine translation methods: traditional machine learning, recurrent neural networks (RNN) with LSTM modules, and a popular large language model (LLM). We evaluate their performance across a synthetic dataset built with proprietary data through different experiments, aiming to identify the strengths and weaknesses of each model, providing insights into their behaviours and effectiveness in terms of practical implementations. The results of this comparative study can facilitate informed decision-making for researchers and practitioners in natural language processing. In this case study, LLMs had a somewhat higher accuracy than RNNs, but at a high additional computational cost.