Jasmine L Shone¹, Robin Liu², Evan Patton², David YJ Kim²*

¹Hawken Upper School, USA
²Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, USA

*Corresponding Author: David YJ Kim, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, USA.

Received: March 21, 2023; Published: April 25, 2023

Abstract

We introduce a cutting-edge learning platform powered by large language models that enables students to effortlessly generate mobile applications for smartphones and tablets from natural language descriptions. We further demonstrate that these user-generated apps can be further optimized with minor adjustments to the generative model's input, or, its "prompt." To maximize the efficacy of the prompt in producing a desired application, we explore three different methods of modification: 1) altering the selection mechanism of example pairs, 2) varying the number of example pairs, and 3) changing the order of pairs within the prompt. The prompts are constructed from a collection of example pairs, which comprise a textual description of an example app and its corresponding code, in addition to a description of the desired app. We test the model's performance by evaluating it with 18 different mobile application task descriptions, ranging from basic to complex, and then leveraging BLEU score to compare the model's outputs to manually created apps. Our findings indicate that the method of determining example pair selection and varying the number of examples included can significantly influence the quality of the generated apps. However, reordering the placement of the example pairs within the prompt does not affect the outcome. Finally, we conclude with a discussion on the potential implications for computer science education. The platform we present in this paper aims to further the democratization of app creation through enabling users to create apps with ease, regardless of their technical background.

Keywords: Large Language Models; Code Generation; Prompt Engineering; Mobile Applications; Computer Science Education

References

1. David Wolber., et al. "Democratizing Computing with App Inventor". GetMobile: Mobile Computing and Communications4 (2015): 53-58.
2. Resnick M., et al. "Scratch: programming for all”. Communications of the ACM 11 (2009): 60-67.
3. Brown Tom., et al. "Language models are few-shot learners". Advances in Neural Information Processing Systems 33 (2020): 1877-1901.
4. Oppenlaender Jonas. "Prompt Engineering for Text-Based Generative Art". arXiv preprint arXiv:2204.13988. (2022).
5. Reynolds Laria., et al. "Prompt programming for large language models: Beyond the few-shot paradigm”. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (2021).
6. Ramesh Aditya., et al. "Hierarchical text-conditional image generation with clip latents". arXiv preprint arXiv:2204.06125 (2022).
7. Rombach Robin., et al. “High-resolution image synthesis with latent diffusion models”. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022).
8. Chen Mark., et al. "Evaluating large language models trained on code". arXiv preprint arXiv:2107.03374 (2021).
9. Gao Tianyu., et al. "Making Pre-trained Language Models Better Few-shot Learners”. Proceedings of the 59^th Annual Meeting of the Association for Computational Linguistics and the 11^th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics (2021).
10. Taylor Shin., et al. "AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts”. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, (2020).
11. Zhang Yue., et al. "PromptGen: Automatically Generate Prompts using Generative Models”. Findings of the Association for Computational Linguistics: NAACL 2022. Association for Computational Linguistics, (2022).
12. Turbak Franklyn., et al. "Blocks languages for creating tangible artifacts”. 2012 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 2012.
13. Zhao Zihao., et al. "Calibrate before use: Improving few-shot performance of language models”. International Conference on Machine Learning (2021).
14. Papineni Kishore., et al. "Bleu: a Method for Automatic Evaluation of Machine Translation”. Proceedings of the 40^th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, (2002).
15. Neelakantan Arvind., et al. "Text and code embeddings by contrastive pre-training". arXiv preprint arXiv:2201.10005 (2022).
16. Radovic Milos., et al. "Minimum redundancy maximum relevance feature selection approach for temporal gene expression data". BMC Bioinformatics1 (2017): 1-14.
17. Zhao Zhenyu., et al. "Maximum relevance and minimum redundancy feature selection methods for a marketing machine learning platform”. 2019 IEEE international conference on data science and advanced analytics (DSAA) (2019).
18. Masson Michael EJ., et al. "Using confidence intervals for graphically based data interpretation". Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Expérimentale 3 (2003): 203.
19. Wolf Thomas., et al. "Huggingface's transformers: State-of-the-art natural language processing". arXiv preprint arXiv:1910.03771 (2019).
20. Fried, Daniel., et al. "Incoder: A generative model for code infilling and synthesis". arXiv preprint arXiv:2204.05999 (2022).
21. Tissenbaum M., et al. "From computational thinking to computational action”. Communications of the ACM3 (2019): 34-36.
22. Smith A. “Older adults and technology use. Technical report”. Pew Research Center (2014).

Citation

Citation: Jasmine L Shone., et al. “Exploring Prompt Engineering for Generative AI-Based App Generation". Acta Scientific Computer Sciences 5.5 (2023): 73-81.

Copyright

Copyright: © 2023 Jasmine L Shone., et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.