I self-host using machines literally under a desk in my home. I've had the setup since 2010.
For sending SMTP, I use the SMTP forwarding host operated by my ISP.
You do not want to be chasing a solution where the same machine that runs all your self-hosted stuff is also sitting on an IP address with a good SMTP-sending reputation.
Have your hosting machine wherever it is convenient for you, and find it a good SMTP forwarding host.
The only downside about forwarding is that the forwarding host or chain of hosts can intercept your plain-text e-mails. The good thing about direct SMTP is that you can use TLS to connect to the target domain to which you are sending e-mail, so you don't need PGP in order to thwart eavesdropping.
However, the security benefit of TLS and direct sending is somewhat negated by most people's mailboxes being hosted by large providers like Google which read their mail.
I think the biggest issue is IP reputation. There are good products out there to do all the heavy lifting, which prevent most of the issue with good default configuration. But no matter how good your setup is, if your IP is/was known for sending spam once, you're on some list with the big providers. If you don't already have an IP for years with a solid reputation, you're going to go through a lot of "please check your spam" communication. I know cases where even neglecting measures such as using DKIM is okay if your IP is okay. The other way around: you need to implement basically every measure possible to even get started being recognized as a trustable sender.
I've had issues with my emails going to spam with Gmail and Outlook, but in my 2ish years of self-hosting email, it never got to a point where it was banned outright. All I do is tell people to check their spam folders for my email and report it as Not Spam, haven't had issues with a ban.
Something that has really helped with rejections and/or spam was having a PTR (reverse DNS) record for your server, some providers like Gmail explicitly require it. It's something that I have to pay a VPS for in order to get. But I can have that VPS with that PTR record and tunnel everything through Wireguard to my home server, all without TLS termination on the VPS.
Stalwart mail is really good and easy to setup. I register a domain name for a few bucks a year and protect it with a Lets Encrypt certificate. Free and very rewarding!!
I self-host using machines literally under a desk in my home. I've had the setup since 2010.
For sending SMTP, I use the SMTP forwarding host operated by my ISP.
You do not want to be chasing a solution where the same machine that runs all your self-hosted stuff is also sitting on an IP address with a good SMTP-sending reputation.
Have your hosting machine wherever it is convenient for you, and find it a good SMTP forwarding host.
The only downside about forwarding is that the forwarding host or chain of hosts can intercept your plain-text e-mails. The good thing about direct SMTP is that you can use TLS to connect to the target domain to which you are sending e-mail, so you don't need PGP in order to thwart eavesdropping.
However, the security benefit of TLS and direct sending is somewhat negated by most people's mailboxes being hosted by large providers like Google which read their mail.
I think the biggest issue is IP reputation. There are good products out there to do all the heavy lifting, which prevent most of the issue with good default configuration. But no matter how good your setup is, if your IP is/was known for sending spam once, you're on some list with the big providers. If you don't already have an IP for years with a solid reputation, you're going to go through a lot of "please check your spam" communication. I know cases where even neglecting measures such as using DKIM is okay if your IP is okay. The other way around: you need to implement basically every measure possible to even get started being recognized as a trustable sender.
I've had issues with my emails going to spam with Gmail and Outlook, but in my 2ish years of self-hosting email, it never got to a point where it was banned outright. All I do is tell people to check their spam folders for my email and report it as Not Spam, haven't had issues with a ban.
Something that has really helped with rejections and/or spam was having a PTR (reverse DNS) record for your server, some providers like Gmail explicitly require it. It's something that I have to pay a VPS for in order to get. But I can have that VPS with that PTR record and tunnel everything through Wireguard to my home server, all without TLS termination on the VPS.
I went down this road and reversed.
I use cloudflare for domain reg, fastmail as my mail provider and free Gmail as my mail client.
Stalwart mail is really good and easy to setup. I register a domain name for a few bucks a year and protect it with a Lets Encrypt certificate. Free and very rewarding!!
I tried using that as well! Are you able to send mail and it not hit spam filters or get rejected?
i just use aws SES, it's already very cheap