— Authored by https://diksha-shrivastava13.github.io/
In a five-membered cross-functional team of a Product Manager, SWEs and an Interaction Designer, I was responsible for the AI features of the product "ai-SAP" for the problem space Wouldn’t it be great if developers at SAP field service management would always know what to do whenever they run into an issue?
The first three months here gave me many of my firsts. It was my first time developing a product from scratch, first time learning the agile mindset of shipping multiple times a week, first time presenting to investors which we did so often, that I eventually got used to just being able to show the product up and running at any time.
The hardest part of this problem was the user feedback loop. How to make sure we know which link is the most relevant to the user problem and proceed to rank it higher the next time a question in the same space is asked? What is the strongest signal?
I was done developing the semantic search and completing most of the stakeholders' requirements at Week 3-4, and it was received a lot better by the users than the keyword-based search that their documentation already had. I had even developed custom query and chat engines. I spent the next few weeks tinkering with ML on user behaviour graphs and determining what can be some useful signals. The cute part about this problem was that the stakeholders were developers, the user were developers, and we were developers, so we had good mutual understanding of how things worked and weekly status.
Since the beginning, we had been taking inspiration from products like Phind.com and Kapa.ai. At the "Rip the Product" session at week 6, the Core Team at DPS pushed us to be bold and brave and afterwards I decided to throw away the entire AI backend. We started building the Internal SearchGPT which can answer user questions from resources scattered across internal documentation, Slack and GitHub with a list of sources and an option to provide feedback from UI as well as natural conversation.
The interesting parts of this are proprietary work which I should not talk about. Some other highlights here were writing many custom modules for chunking, 15+ readers and ~13 LLM calls system, parsing and metadata augmentation, integrating Mlx into LlamaIndex, hosting private LLMs, the continual augmentation of answerable questions which aligns the knowledge base closely to user requirements.
I gave a two-hours long technical presentation during the product handover detailing every part of the code, every technical decision, all the whys of the models I used, scope for future development, all the custom augmentations I had made. That presentation was one of my brightest experiences of 2024.
DPS deserves a different section since even outside the products I worked on with SAP and BMZ (in partnership with DPS), this was a different experience. The Digital Product School is a training program by UnternehmerTUM, the largest start-up ecosystem in Europe.
They form product teams of 4-5, with a Product Manager, an Interaction Designer, SWEs and an AI Engineer and give them a wide problem space where they are supposed to conduct user interviews, identify pain points, identify which are the most critical problems for the user that they can solve in 3 months and start rapid prototyping. Afterwards comes the phase of conducting user tests, iterating on feedback and more rapid prototyping.
Since it's in a very active start-up environment, we get multiple opportunities to pitch to investors and receive feedback. All my major "news" about the pitches made to people at Meta, SAP, IBM, Seroton, MTZ, AWS, UnternehmerTUM and United Internet Media GmBH has come through the time spent in this program. The best thing about DPS is that no one tells you how to do things or what are the right directions, and you are supposed to figure everything out on your own. That's the biggest learning. They treat you like an independent start-up team with an advisory board and stakeholders. And the stakes are real.
I have met the nicest, sweetest people here who were always ready to discuss and share wisdom, every achievement was widely celebrated and every setback opened a whole new learning path. The people at DPS are extremely nice, it's a little hard to process that people can be that sweet, especially as they gift you The Mom Test: How to talk to customers & learn if your business is a good idea when everyone is lying to you on the first day and recommend the Blue Bible (The Lean Startup) later. And trust me, users lie all the time, no one wants to be mean. As an undergraduate, I cannot imagine a better place to have learnt engineering.
There was way too much compressed learning in a short duration of 3 months when I was on-contract with SAP. The only experience I had with generative AI before this had been finetuning a model for a hackathon by Complex Systems Lab, IIIT Delhi, using Cohere's API. Two months in, I was customising LlamaIndex modules to perform on queries from 100,000+ users.
I was actually very nervous when I joined, considering I was the youngest in my batch and almost everyone else was a Masters student. My biggest fear was that while there's a good enough chance I would be able to build the expected level of amazing, which is still an achievement, but it does not blow your mind and that way more learning is required to leap to the unexpected, people-can't-stop-talking level of amazing. And it gave me many sleepless nights.
Fun Fact: After I threw away the AI backend, and we decided to pivot to building internal Perplexity/SearchGPT/Phind.ai, the Chief Strategy and Operating Officer of SAP, Sebastian, was visiting the Munich office and our co-working site, and so there were a lot of SAP employees in the building. I was in the kitchen getting chai when I heard someone talking about there's this team here, and they're doing crazy stuff.... Cue * happy, little hops *.
It was actually wild to see how excited people were about the product we built for SAP! I have heard "they're doing crazyyy stuff" a lot of times in the final few weeks of product shows and that we crossed all expectations that were there before us in the history of DPS and set the bar much, much higher.
In hindsight, that fear of maybe not being good enough actually worked wonders.