I’ve been a Software Engineering Student for 2 years now. I understand networks and whatnot at a theoretical level to some degree.
I’ve developed applications and hosted them through docker on Google Cloud for school projects.
I’ve tinkered with my router, port forwarded video game servers and hosted Discord bots for a few years (familiar with Websockets and IP/NAT/WAN and whatnot)
Yet I’ve been trying to improve my setup now that my old laptop has become my homelab and everything I try to do is so daunting.
Reverse proxy, VPN, Cloudfare bullshit, and so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
Why is self-hosting so daunting? I feel like even though I understand how many of these things work I can’t get anything actually running!
You must log in or register to comment.
Part of working with tech is knowing what to search for und using the right keywords. If you could give an example what guide / information you are unable to find, someone could give you an example how to search for it. I personally know a few junior devs and junior devops which use llama2 / chatgpt since they dont know how to search for it or read the docs.
I actually hate GPT, dislike it’s answers and find myself knowing better than it most times.
I’ve been trying to setup a DNS server to create my own domains internally within my VPN but I keep finding info on how DNS servers work, and how to make a records on registrars, but nothing on what I actually need to install and run to have my own DNS for example. Same thing goes for many other services, but that’s the one bugging me for the longest time because it should be so simple.
I’ve found plenty of tutorials on how to make a cache DNS, just not an authoritative name server btw, and I’ve searched for both DNS and name server to no avail. If it was Linux I’d write some custom rules in my hostfiles and be done with it, but it’s so much harder to do on Windows and that’s my daily use OS for now…
I hate the fact you gave your honest opinion and othets have to downvote you. I myself rarely use ChatGPT to find a direct answer, instead I use it to give me resources to find the answers for the questions I didn’t think about asking or knew existed. More like thinking outside the box or brainstorming. And always do your due diligence to verify any answer it gives.
“Selfhost authorative dns” returns this for example https://wiki.selfhosted.show/DNS/
And we have two guides for the most common dns implementantations. Depending on the vpn software you can push the dns ip as part of the configuration or by using DHCP.
I also found the pihole docs for unbound rather helpful https://docs.pi-hole.net/guides/dns/unbound/.
For me, I have that as the 4th result, after some Reddit and IBM which probably would’ve discouraged me from continuing my search. I’d have to read on it.
Also, TIL PiHole doesn’t necessarily need to run on a Raspberry Pi. I guess assumptions really do come back to bite me in the ass haha
Pi runs Raspbian which is just Debian with customisation applied. So of course it can run elsewhere. You don’t know as much as you think you do perhaps 😉
I seriously thought it was a product, rather than software tbf. The name always sounded so “corporate” I never considered it.
I definitely know more about the theory than the practice. I’m clueless as to what my options even are so I can’t argue with that.
But I did know about the Linux “inheritance” of distros if you wanna call it that, and I’m fully aware of what that entails.
Just honestly didn’t look at it twice cause I thought “there must be an FOSS option” without realizing what PiHole really is. Just a case of prejudice biting me in the ass I guess.
Sounds like the next step in your journey is combing through this list and seeing what’s out there: https://github.com/awesome-selfhosted/awesome-selfhosted
So much great stuff! But most of it has drawbacks, like missing features or less attractive UI. But it’s free and open source so we love it all the same.
I’ve read that repo a million times! My self-hosting needs are more esoteric and I mostly play around with it. I’ve no need for media services or 90% of what that repo offers yet!
I mostly want to end up self-hosting my own apps, but I need some foundational knowledge
There’s a lesson in here somewhere about patience. Get good at skimming. I was looking for how to do something I was unfamiliar with the other day and I had to sift through 15 results across four different search strings before I found the solution that was going to work for me. But because I’m good at skimming it only took me 1/2 hour to discover and implement. Google isn’t magic and this is why someone else recommended Chat GPT to help with some of the sifting, especially early on.
I use Pihole for this in my home network. It has the capability to configure local domains for your internal network resolution as well as create whitelist/blacklist and use as a DHCP server alternative to your home router – you just need to configure your router to point to the Pihole instance for DNS on your LAN network configuration to get started with the DNS piece
“self hosted DNS server” is what I searched for, but I also knew what I was looking for “Unbound” with PiHole is what I use.
Also, Windows has a host file that it uses. %windir%\System32\Drivers\etc\hosts
%windir% is the environment variable for your windows install directory. Usually C:\Windows
You can add your entries in there, just like Linux, though I am sure the syntax might be slightly different. I really haven’t played with host files on Linux, yet.
Each one of these things is a separate topic/project. There is of course, overlap, but concentrate on just one at a time. If you try to do everything, you’ll go mad. Application Administrator, Network Engineer, Sysadmin, Network Security, are often separate full time jobs. Just concentrate on your own needs, one at a time, then focus on how simple you can pare down that one thing you need/want to self host. What is the minimum set of capabilities you must have for this one thing? Then, expect to have as much research for this one thing as taking a 1 credit college course, at least. Maybe more, maybe less, depending what it is. That’s a more realistic set of expectations.
I’ve done a lot of web design and web hosting, and messing with Linux server stacks for around 30 years, now. You absorb a LOT over time, but it is all always changing. When I set up my first server, Cloudflare didn’t exist yet. VPNs weren’t a thing yet. The only people I knew with a network in their home were Computer Science professors. Wifi was not a thing yet. It isn’t you being inadequate, it’s you trying to do multiple jobs that each require constant professional development.
Check YouTube. I haven’t found a lot of written out guides but someone has made a video on it, I bet. Christian Lempa, Techno Tim, Db Tech, NetworkChuck come to mind for self hosting tutorials on containers and new applications. I’m sure there’s a lot more as well.
Thanks for the suggestions! The algorithms keep feeding me people who just explain what stuff is and it drives me nuts. You wouldn’t believe the amount of videos, articles and blogs I’ve seen on setting up a DNS server just for it to be about either a cache or an explanation of how it works. I’ll look into these later!
Jeff Geelong is another good resource on youtube, especially if you ever want to get into infrastructure as a career.
Github is also an amazing resource and always RTFM.
Edit: spelling
I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
This is big part of why I, even as an IT professional, don’t do much homelabbing or selfhosting. So often I’ll be scrolling through these subs and see something someone has done and I’ll decide I want to do that too, but so often I end up abandoning whatever it is because doing x requires googling for y and z because without y and z, x doesn’t work. And after a day of dealing with issues and tinkering on work stuff, do I want to spend more time troubleshooting? Often not.
Maybe you should read the documentation for the programs you try to install
Because most people who develop these things are, frankly, terrible at good documentation, or understanding the end-user perspective.
There’s also a downward spiral effect when you start getting into these things, because lots of them require dependencies, or ask you to do things but don’t explain why, and you’re just left wondering why you added that line to a config file somewhere, but if you don’t put it there, nothing works.
A vertical slice of the amount of knowledge you need passes through so many different disciplines, operating systems, GUIs, and programming languages that it would look like a Milhojas cake.
I’ve been a technical writer in the software industry for 17 years. The number one challenge in my work is extracting all of the information I need to write good documentation from the experts elsewhere in my company.
You’re kidding?
I’m an avid home-labber trying to get into software engineering and I’m finding the software dev work to be super daunting lol
I can setup a reverse proxy in my sleep - but write a JS function without double checking google? forget it!
It comes with experience I guess, I’ve got a bad habit of researching to the core and many times have a hard time grasping things like containers without understanding how it’s setup technically. Sometimes I find a decent explanation, but specially for libraries that do “magic” I gotta go diving into the source to understand what’s going on, else I have trouble understanding what I am doing and what I should be doing.
Which makes it so hard because networking is very low level and I’m very unfamiliar with this environment
NetworkChuck on utube
You don’t have to do it, 99.9999%^([citation needed]) of the population don’t. If it doesn’t interest or excite you then it’s not worth it.
Its only daunting if you want to do everything from scratch. Depending on your needs you could easily build off an existing platform for home labs and servers. Its more of a tinker / experiment / use existing, then reverse engineer the parts you need to.
A good article on Home Server OS is here:
https://www.smarthomebeginner.com/best-home-server-os-2023/You can Find some good Self-Hosting projects / apps here:
https://selfhosted.libhunt.com/And finally, here are two examples of decent, simple to install all-in-one solutions, Yunohost is my go to for friends who want to start self-hosting as it sets up everything including SSO, DNS (via Adguard Home) and Certs via Lets Encrypt all in a nice web gui and admin panel.
Yunohost:
https://yunohost.org/#/HomelabOS:
https://homelabos.com/I would say the opposite: self hosting nowadays is very easy. Is the multitude of options and configuration possibilities that is daunting. We want too much and this increases complexity.
As someone who has been self-hosting stuff on and off since middle school, I feel like nowadays the choice paralysis is much much worse. Oh, and the sheer amount of outdated advice and info is a big issue.
I’d say, start with a need and not with a tool. Tools are much easier to understand once you know the problem you have to solve.
I’d just install Docker and add a CloudFlare Tunnel to securely host your stuff. No need to open any ports, your WAN IP remains hidden and you have the benefits of DDoS protection. It’s free as well; all you need is a domain name for which you can configure the nameservers.
Putting a server in front of my own defeats the whole purpose of self-hosting for me.
I didn’t say CloudFlare “bullshit” so aggressively for no reason.
I want to learn, because I feel like I should know how to deploy stuff and my uni is not teaching me.
Self-hosting is like any other topic in computing. You have to read, read and read. If you want to self-host a reverse proxy, go to the official documentation of the reverse proxy. What you are doing is searching for magic recipes in blogs and YouTube, that is why you feel it so “daunting”.
Quite the contrary, I’m stuck at finding a reverse proxy in the first place. If I didn’t know nginx had a reverse proxy, which is the only one I know about, where would I even start finding the docs? I can’t repeat this enough, but I rarely ever do tutorials, I find them basic and lackluster
I don’t think I need specifically a reverse proxy rn so I don’t really have a clue about that kind of service specifically, but even finding WHAT to use to do an authoritative DNS was a challenge in its own right that I only solved somewhere else in this thread.
This is a swiss army knife of hobbies. I would say that yes it can be hard to get into if you try to do everything at once. I would pick one to three goals you want to accomplish and do those items. Once you have those documented and implemented, then pick up another task.
The amount of times I’ve completely rebuilt my home setups is… many. But to get where I am now, I would never have been able to plan it properly without the learning experience of “failing” so many times. And there will always be a better way to do it as things need upgrading so in a way it is perpetual. Start with the projects you want first. Get them working and make sure you can export the data properly before you begin to rely on them. Then once you have small bits done, add extras, like cloudflare, vpn, portainer, proxmox. There will be a lot of formatting and reinstalling and going back to the drawing board, just accept it as a learning experience but take each piece once by one. If there is a specifc
yet I can barely find info on HOW to set up this things.
There are plenty of guides out there that makes assumptions that you should already know things, which can be frustrating, I found this especially true with docker related documentation. If you find yourself on a github page, check the Releases tab. If it’s still confusing there, check for a docker folder in the github and search "docker ". Start with popular more documented projects that will usually include beginner instructions that less popular software will omit. Even then if you keep looking you can usually fine one good doc that clears things up.
Also, in this realm, chat GTP excels at obscure configurations so it’s worth giving it a shot when stuck.