Hello, not so long ago I discovered Jellyfin, and by extension Radarr, Sonarr, Bazarr, Jackett, etc.

So I immediately installed Jellyfin on a Hetzner server that I already had, powerful enough to have several people consuming content at the same time on these kinds of services.

As the hard drives at home were full, and I didn’t want to sacrifice my bandwidth, I got a Hetzner Storage Box to store the video content on.

So I’ve got Radarr, Sonarr and qBitTorrent installed on my computer, where I manage the downloads, upload them to the Storage Box which is then mounted on the server where Jellyfin is installed to serve the content (see the diagram above, the orange part would be a hypothesis for integrating more storage in the future).

Problems :

  • Synchronisation between my local machine and the Hetzner Storage Box: as I was saying, I download everything locally and then upload everything with rclone sync to the storage. The exact command is: rclone sync /mnt/2TB/Jellyfin HetznerBox:/Jellyfin --exclude "*.!qB" --transfers 10 --fast-list --checkers 500 --progress --log-level DEBUG --log-file=/var/log/rclone/rclone-sync-jellyfin.log This has several problems, it’s slow, and I have a 1 GB/s fibre connection, and I doubt the problem is with Hetzner… Secondly, if I delete content on the storage server and run the command again, the content will be resynchronised. What’s more, it often happens that I run the command and nothing happens for a long time before the synchronisation starts. Thirdly, rclone sync forces me to keep a local copy of everything.
  • rclone mount does not seem to respect the --vfs-cache-max-size: here is the command I use to mount the storage server on the server where Jellyfin is installed rclone mount HetznerBox:/Jellyfin /somepath/Jellyfin/ --vfs-cache-mode full --vfs-cache-max-size 100G --allow-other. But if I run df -h I get /dev/md2 436G 148G 266G 36% / And if I run sudo du -sh Jellyfin/ I get 378G Jellyfin/ The server only has Jellyfin installed, and I don’t know how to look precisely how much the vfs cache is using.
  • since Radarr, Sonarr and everything are on my computer, if I install Jellyserr on the same server as Jellyfin (which would be perfect for managing everything when we have several users), it won’t be able to communicate with Radarr and Sonarr.

Possible solutions:

  • Don’t use my local machine. I’d like to use the Hetzner server for Radarr, Sonarr, qBitTorrent, etc. but the problem is that I don’t want to store the data on this server, but on the Storage Box. How can I download directly from one server to another?
  • Could I use Mullvad VPN only for qBitTorrent downloads (and Jackett) on a Hetzner server, or is that too complicated or foolish?

Edit:

Diagram updated to adapt more to my configuration:

image

  • the Hetzner servers are in Finland
  • tried again overnight, rclone mount works fine, just extremely slow
  • downloads of “finished” torrents and those “in progress” are already separated
  • Katzastrophe@feddit.deEnglish
    15·
    1 year ago

    Can’t help you, but I have to ask, why Hetzner?

    They are a german server hosting company, you know, the country with some of the strictest copyright laws? For example, a kid at an old school of mine was once fined 800€ for accidentally sharing a song on a school hosted messaging board.

    If you do anything with pirated torrenting on their servers, they will know, and will come after your ass. And even sharing pirated content can get you in incredibly hot water, they might not have found out yet, but when they do, good luck.

    • bier@lemmy.blahaj.zoneEnglish
      3·
      1 year ago

      I’ve been using a Hetzner server for a couple of years now and haven’t gotten any abuse reports or DMCA takedown request but I’m using a docker + vpn setup so maybe it’s that

    • retiolus@lemmy.catOPEnglish
      1·
      1 year ago

      Hetzner because that’s what I had and have always had a good experience with them, anything else to recommend at the same prices? And that is a reason why I don’t download the content directly on their servers but on my local machine.

      • Katzastrophe@feddit.deEnglish
        5·
        1 year ago

        The only one in a similar price range that comes to mind is ovhcloud.com, they are a france based company, still not ideal though, as they might require ID.

        If money is the biggest factor (which I guess it is), best thing for you to do, might be to move your storage box off of the german servers and onto the finnish servers.

          • taaz@biglemmowski.winEnglish
            11·
            1 year ago

            Even in the finish location, if any of the “law” fuckers scourging the open trackers, sends an abuse/dmca to Hetzner, you will have some explaining to do as they will raise it as a ticket and require an explanation as to what happened and what is your course of action to remedy it.

            Source: my own experience about two months ago (used chatgpt to send them a neutral bussiness respone, worked like a charm)

            Use vpn that allows port forwarding, or never touch public trackers at all, or both.

        • VonReposti@feddit.dkEnglish
          1·
          1 year ago

          I don’t think I’ve ever given my ID for my Kimsufi or SoYouStart servers. Both of them are OVH subsidiaries.

          • Katzastrophe@feddit.deEnglish
            1·
            1 year ago

            I’ve heard they do it sometimes for high risk customers. Though they might have stopped some years ago

      • Rescuer6394@feddit.nlEnglish
        21·
        1 year ago

        they can detect piracy regardless. generally they set up honeypots and if you download a torrent from them, they will know and fine you.

  • tmax@lemm.eeEnglish
    9·
    1 year ago

    Have you tried Prowlarr instead of Jackett ? It’s really really nice !

    • Lupec@lemm.eeEnglish
      5·
      1 year ago

      I second this, Prowlarr is way smoother to use in my experience

  • whyNotSquirrel@sh.itjust.worksEnglish
    8·
    1 year ago

    but the problem is that I don’t want to store the data on this server, but on the Storage Box. How can I download directly from one server to another?

    Can’t you mount the Storage Box on the Server? Some other comments says that they allow SMB?

    • retiolus@lemmy.catOPEnglish
      1·
      1 year ago

      I only heard that it was not advise / that it did not work well, and I actually tried it myself instead of using rclone sync to upload the content to the storage, mount it on my local machine. qBitTorrent downloaded everything, but the content was never reflected on the storage.

      • Red@reddthat.comEnglish
        2·
        1 year ago

        You might have setup something wrong then. It should work. The problem with torrenting on 1 server and saving the data to another directly via smb/NFS is that every write is at the behest of uploading to that network storage.

        What would be better might be that you:

        • add categories
        • mount /completed/tv/ as a network share to hetzner-storage
        • then only when the torrenting is done does it auto copy the data to the SMB share

        I have this all done internally. I torrent on one box, it’s finished and gets moved via NFS to the storage server. Which sonarr/radarr file away. They notify jellyfin to resync the library. The jellyfin box has the storage box mounted via NFS.

        If you can test with small files, IE copy a 100m file onto the SMB share and then see if that was replicated to the storage box then you will have the basis of the solution.

  • newIdentity@sh.itjust.worksEnglish
    5·
    1 year ago

    I’m not really familiar with the Storage Box, but since you mount the storage box on the Hetzner server anyway, why not download directly to it?

    Apparently it supports SMB so you could use SMB instead of rclone and mount the drive with fstab, but I don’t know if it’s faster

    • retiolus@lemmy.catOPEnglish
      1·
      1 year ago

      I only heard that it was not advise / that it did not work well, and I actually tried it myself instead of using rclone sync to upload the content to the storage, mount it on my local machine. qBitTorrent downloaded everything, but the content was never reflected on the storage.

      • newIdentity@sh.itjust.worksEnglish
        31·
        1 year ago

        Another solution would be to set up a reverse proxy for your local network, but I don’t know if it isn’t a little overkill.

  • 7Sea_Sailor@lemmy.dbzer0.comEnglish
    61·
    1 year ago

    Why go all this way to build this stack on German hardware under the constant risk of being found out by copyright holders and being fined hundreds if not thousands of euros - when you could just get a seedbox that’s automatically configured for exactly this purpose and can run all the apps you need? Look at ultra.cc for example, they just work, don’t cost a lot of money and don’t annoy you with DMCA notices. And if you dislike their price or anything else, there’s limitless options out there.

      • 7Sea_Sailor@lemmy.dbzer0.comEnglish
        3·
        1 year ago

        Ultra is what I personally have been using for a while now without a single problem. But as I already said, there are plenty of other hosters with the same featureset, though I cannot make specific recommendations - but the seedboxes community on reddit may.

        Regarding the DMCA, usually these hosters base their operations out of countries with lax or nonexistant copyright laws, allowing them to simply disregard incoming takedown notices. In Ultra’s case, they claim to forward copyright notices to the respective server owner, but I have yet to receive a single notice. Though I’m also exclusively on private trackers, maybe thats a reason too.

    • ShortN0te@lemmy.mlEnglish
      1·
      1 year ago

      How should they even know? How should they prove that you are in posession of that data? Yes Hetzer most likely has to respond to reports but again how should the rights holder find out? Just because they find a jellyfin server in the wild? Even if the site is not SSL protected they are not Man in the Middle the traffic.

  • pcjones@feddit.deEnglish
    3·
    1 year ago

    I also use a Hetzner Storagebox for my Jellyfin instance, I mount it via SMB. Works great.

  • killabeezio@lemm.eeEnglish
    2·
    1 year ago

    Not sure where you got the idea that it’s not advisable to mount the box via NFS. You can totally do this. I would make some adjustments though.

    I would use mergerfs to union multiple mounts into one. You would then download to the local mount which is the drive connected directly to your seed box. Then I would have a remote mount to the nfs mount. You merge these into one so that when you link up jellyfin, it won’t know the difference and you can just stream like normal.

    You need to copy files from the local drive to the remote, so you can try and roll your own solution by using rclone or use something like cloudplow which solves this issue as well. Cloudplow uses rclone as well, but monitors for changes automatically.

    As far as copying files, why are you using sync anyway? It’s pretty dangerous. Just use move or copy instead. This way you don’t need to keep copies on your computer and the server.

    As far as streaming from the nfs mount. You may need to make some changes to the cache settings and ensure they are set correctly.

    With a setup like that, you should have no problems though.

  • swallowyourmind@geddit.socialEnglish
    0·
    1 year ago

    I would recommend you look into Saltbox, which seems to align with what you are looking to do and should take considerable heavy lifting off your plate.

    The solution to the rclone issues you discuss is to use a union mount, and cloudplow or similar to automatically shift files to the long term storage location, while Jellyfin will continue to see content files spread around servers in one mount location.