So to play the devils advocate, this really comes down to:
- Are these internal people or contractors
- Are they looking to test out a Workflow or implement small automations
If they’re contractors, always be suspicious. If they’re internal, it sounds like they may want a server so it’s a one off cost to setup some work flows. You can burn through hundreds to thousands of dollars per month using LLM APIs, but if you self host you’ve basically got a fixed cost.
If they’re just looking to create some work flows that crawl the data dumps and give some insights, this approach could work. Although, as others mentioned, the hardware they’re talking about would be somewhat limited. Small LLMs can be quite functional, but this server might not be a long term fix if the automations they set up gain traction.
Yeah, then that’s a weird ask. They could have a legitimate plan, but it’s weird to not start with a mvp using an API or similar before buying dedicated hardware. If they can’t articulate what they’re going to do, then they shouldn’t know what hardware they even need.