Useful archiving efforts and other projects to help out with for people new to and interested in archiving:HIGH priority (If you don't help archive these automatically, the data will probably be lost forever):1. http://warrior.archiveteam.org/Help out automatically archive things being shut down right now by running ArchiveTeam Warrior program (or specific containers) in the backgroundRequirements: Few GB of space, some bandwidth and small amount of CPU power, more info: https://wiki.archiveteam.org/index.php/ArchiveTeam_WarriorIf you learn that a site or any online data is in danger of shutting down, read through this page and contact ArchiveTeam on their IRC if required in order to have it archived: https://wiki.archiveteam.org/index.php/Projects2. Help out automatically forward URLs you browse that are not archived on https://archive.org to them for archival with a browser extensionhttps://github.com/internetarchive/wayback-machine-webextension
MEDIUM priority (Important overall)3. Seed torrents for as long as possible, rare data forever. Make sure to look up a guide for your router to PORT FORWARD your torrent client port, to substantially increase your upload (and your download) speed. In low population torrent swarms, if no one is port forwarded then you might not be able to connect to each other at all and exchange any data despite having it.Requirements: As much or as little bandwitdh you want (you can set the limits if you need to)https://github.com/qbittorrent/qBittorrent (Recommended client, especially to replace uTorrent)4. Archive web pages you want to have a local copy of with a "Web Extension for saving a faithful copy of a complete web page in a single HTML file with a single click"https://github.com/gildas-lormeau/SingleFile5. Archive videos with "GUI front-end for youtube-dl, yt-dlp and other compatible video downloaders"https://github.com/axcore/tartube6. "Capture or record any area of your screen and share it with a single press of a key"https://github.com/ShareX/ShareX7. Archive entire websites you want to have a local copy ofhttps://www.httrack.com/
8. Publish the data that you have archived that isn't easily or at all available online. You can easily create torrents yourself in your torrent client and then share the magnet link to it anywhere online for anyone to access and, as long as DHT (Distributed Hash Table, decentralized way to share torrents without the need for any specific tracker) is enabled in settings (on by default), your files will be searchable on DHT by DHT crawlers, local or online (for example https://btdig.com/, where you can actually also search for FILE NAMES within all DHT torrents)(archive.org also creates torrents for all uploads automatically but their torrents shouldn't be relied on because of an error-prone implementation and since they can also break when more files are uploaded or if the item's metadata changes, which includes even getting a new comment on the item)OTHER useful things:- In your torrent client settings add the best trackers to be automatically added for all of your newly added torrents (helps more easily connect to peers, especially in obscure torrents)https://github.com/ngosang/trackerslist- If you have a torrent in your torrent client which is stuck at downloading metadata, you can see if the .torrent file containing the metadata was cached online. Copy the v1 torrent hash of your torrent and replace it in the link below which uses itorrents.net and btcache.me services to check:https://downloadtorrentfile.com/hash/c8295ce630f2064f08440db1534e4992cfe4862a- Look into running a node for I2P (anonymous private network within the global internet)Requirements: Mostly bandwidth, more info: https://geti2p.net/en/faqhttps://geti2p.net/- Look into running Tor/Hyphanet(Freenet)/IPFS/YaCy/SearXNG nodes- Easily capture and digitize all data AND METADATA from optical media (CDs, DVDs, Blu-rays...) with Media Preservation Frontend (MPF)https://github.com/SabreTools/MPF
- "A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI"https://github.com/bitmagnet-io/bitmagnet- "ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline"https://github.com/ArchiveBox/ArchiveBox - Look into donating your PC resources to be used more intensively in projects:BOINC (Berkeley Open Infrastructure for Network Computing): https://boinc.berkeley.edu/projects.php- Additional archiving tools: https://github.com/iipc/awesome-web-archiving- Additional links to archiving and similar communities:https://wiki.archiveteam.org/index.php/Archiveteam:IRChttps://www.reddit.com/r/Archiveteamhttps://www.reddit.com/r/DataHoarderhttps://www.reddit.com/r/DataHoarder/wiki/index/ - Hardware and software for data hoarding FAQhttps://www.reddit.com/r/lostmediahttps://www.reddit.com/r/GamePreservationistshttps://www.reddit.com/r/torrentshttps://www.reddit.com/r/qBittorrenthttps://annas-archive.se/torrents>>>/t/What are you archiving or want to archive?Do you have or know anyone who has some rare interesting data or media not available online?
Added last:>- If you have a torrent in your torrent client which is stuck at downloading metadata, you can see if the .torrent file containing the metadata was cached online. Copy the v1 torrent hash of your torrent and replace it in the link below which uses itorrents.net and btcache.me services to check:https://downloadtorrentfile.com/hash/c8295ce630f2064f08440db1534e4992cfe4862a
>>110206580Can someone get these fuckers to finally merge tmpfs support for their container image? No, I won't use some random fork or maintain one myself.https://github.com/ArchiveTeam/warrior-dockerfile/pull/90I uploaded a couple TBs but stopped once storage prices got retarded. I have 8 residential IPs just begging to be used.
>>110206730Make a ramdisk and put the software there, if it just works on windows it must be easy to set up on linux too
So what’s the easiest way to download videos from ThisVid?
>>110206931yt-dlp
tldr
>>110207318http://warrior.archiveteam.org/
>>110207119>>110206931How do i get yt-dlp to add the author/uploader etc to the filename if %(uploader)s, %(channel)s, etc don't work?Is there a way I can manually tell yt-dlp, "hey, this bit of html/page source/whatever is where the text field in question is"?Same for if it's not detecting the video to rip at all but I can find the mp4 url in the page source/elements?
I need a portable HDD to plug into my PC's via USB, but everything is overpriced right now and I want to buy a refurbished or used internal HDD instead and just use a USB enclosureHowever, the only drives I can find for a really good price are SAS rather then SATA, and I can't seem to find a enclosure that will work with SAS drives that would function like a portable drive that's not obscenely overpriced, unless I'm just not looking in the right places.Do they not exist or what?
>>110207640The easiest way is to ask any online free AI to read the docs and give you the command or the steps to change the config.If some things are not possible then the AI can create a script to scrape the web and rename the files at the end.For sites where yt-dlp doesn't work, the easiest thing to try is Video Download Helper extension.
I am still looking for altchan archives including the ones that can't be mentioned.
>>110207722I don't use AI tools, like, at all.So I wouldn't even know how to find one to ask it this stuff. And I'm not giving an AI company my email address and phone number to sign up (the one time I tried to sign up with ChatGPT, it required both, so I couldn't just use a throwaway email service)>>110207875I don't think you get banned or have your posts deleted for mentioning 8ch anymore, I haven't seen that happen in many, many years. Anyways I don't know of any archives for any other imageboards, sorry anon
Not sure where else to ask this:Why do so many albums on bunkr have less (or even zero) images or videos when you open them compared to the amount of files it said they had in the search results?There's some albums that in the search results were marked as containing hundreds of images that then only have a few when I click the album to view the files, and tons of them straight up just have no files after opening them even if it said otherwise in the search resultsIs this from DMCA takedowns or something? Is there any way to view the original set of files a album had just in terms of filenames even if the video/image isn't viewable?
>>110207640>>110207884You could try modifying the extractor script, it's Python.
>>110207884You should be able to ask ChatGPT a few questions without registering, if you don't have a single burner gmail then get into local models https://owo.vg/g/lmg
>>110207935If you have a guide on how i'd do that, I'll give it a look, but the extent of my coding/programming/scripting knowledge is occasionally stumbling my way through using command line tools like this to begin with, and even then I struggle with them
>>110207888bunkr servers are down a lotor maybe indexing wasnt updated
>>110207946There's no way of doing it through the CLI.I don't have a guide or specialized experience for writing extractors, it would also depends on the website I suppose, sorry.Python is worth learning in general is you're doing scripting often though, it's way better than shell.
>>110206580>archiveteam.orgWhy are these faggots so incompetent?They seems to archive (badly) the most irrelevant shit on the internet while all the good stuff is routinely lost, are they a psyop?
>>110208075You're overestimating my knowledge, I don't even know what "shell" is
>>110207875>>110207884https://archivohot.net/ seems to be an archive for a altchan
>>110208479also this? https://arhivach.vc
>>110208265>while all the good stuff is routinely lostlike?
>>110210963Porn?
>>110211846private trackers should have enough
>>110213924Private trackers are not public archive, in fact they will be one of the reason why media will be lost, all because some faggots want to feel like they have authority and powertrip
>>110207655Well, I found this:https://www.amazon.com/OSENTRiX-GX200S-External-Enclosure-Driveless/dp/B0HLR9JDZSAnybody familiar with this brand?