Over the past three weeks, I encountered several technical challenges while working on my Sauer Studio A.I. project. If you haven’t read my earlier posts, in 2024 I created an open-source generative AI server to support my research, because the largest number of existing genAI servers today this sort of setup (though most published research has focused on cloud-based services like ChatGPT, Claude, or Gemini). Too few people are even aware of how capable open-source alternatives already are, I feared.
Exploring the Open-WebUI Interface
As part of my studies, I experimented with various settings in the Open-WebUI front-end web server, to optimize my genAI models’ performance. These included:
- Turning administrative features on and off, such as “Tags Generation,” “Autocomplete Generation,” “Retrieval Query Generation,” and “Web Search Query Generation.”
- Adjusting model parameters like context size (short-term memory), temperature (creativity level), and max_tokens (response deliberation time).
- Upgrading the Open-WebUI application multiple times.
- Testing whether allowing the genAI to search the internet or preload specific documentation sets would improve results.
Performance Decline and Troubleshooting
Despite these efforts, the system’s performance deteriorated over time. Responses became increasingly slow, and answers to previously correct questions became vague and less specific. I tried numerous fixes:
- Disabling the Content Moderation Filter.
- Adjusting context window sizes and token settings.
- Reinstalling older versions of Open-WebUI.
Nothing worked. I felt discouraged, questioning whether the system had ever performed as well as I initially believed.
Additional Setbacks
At the same time, my open-source Certbot (which generates SSL certificates for my websites) crashed without clear error messages, affecting all sauer.studio domains. Coupled with disturbing national news, I was tempted to doubt myself entirely, including my ability to manage a modern container-based AI server (even after 30 years of web hosting experience).
I felt overwhelmed. I felt guilty about this failure’s implications for my first research study’s coauthors, and for the 28 scholars worldwide who’d created accounts on Sauer Studio A.I.
Solution Found
After reinstalling the newer Open-WebUI version and extensive research, I finally resolved the issue by turning off the four administrative settings mentioned at the top of this post. This restored the system’s performance, and I quickly fixed the SSL certificate problem as well. Now, everything runs smoothly, just in time for user testing.
Lessons Learned
This experience coould teach me one of two lessons:
- Perhaps it’s better to use established cloud-based services like ChatGPT instead of building local alternatives. While this approach may have ethical concerns and limited customization options, it would probably offer stability.
or
- Academics need to thoroughly explore AI server settings and configurations before publishing research. Hands-on experimentation is crucial for understanding these technologies fully.
Future Plans
I’ve chosen the second lesson as my guiding principle. I believe that mastering new technologies requires extensive exploration of their underlying infrastructure. Moving forward, I plan to enhance our project with more resources and personnel. It’s only through trial and error that I now truly grasp concepts like “tags generation” or “retrieval query generation.” This hands-on approach has deepened my understanding of LLM genAI technology.
I suspect all scholars who study generative A.I.s would do well to spend detailed time learning everything we can about these technologies, so we’ll be able as a community to help our discipline think about how to professionalize any widespread use of emerging technologies like these.