[a / b / c / d / e / f / g / gif / h / hr / k / m / o / p / s / t / u / v / vg / vm / vmg / vr / vrpg / vst / w / wg] [i / ic] [r9k / s4s / vip] [cm / hm / lgbt / y] [3 / aco / adv / an / bant / biz / cgl / ck / co / diy / fa / fit / gd / hc / his / int / jp / lit / mlp / mu / n / news / out / po / pol / pw / qst / sci / soc / sp / tg / toy / trv / tv / vp / vt / wsg / wsr / x / xs] [Settings] [Search] [Mobile] [Home]
Board
▼ Settings Mobile Home
4chan
/g/ - Technology


Thread archived.
You cannot reply anymore.




File: 1782845697948194.png (337 KB, 1497x936)
337 KB PNG
Useful archiving efforts and other projects to help out with for people new to and interested in archiving:

HIGH priority (If you don't help archive these automatically, the data will probably be lost forever):

1. http://warrior.archiveteam.org/
Help out automatically archive things being shut down right now by running ArchiveTeam Warrior program (or specific containers) in the background
Requirements: Few GB of space, some bandwidth and small amount of CPU power, more info: https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior

If you learn that a site or any online data is in danger of shutting down, read through this page and contact ArchiveTeam on their IRC if required in order to have it archived: https://wiki.archiveteam.org/index.php/Projects

2. Help out automatically forward URLs you browse that are not archived on https://archive.org to them for archival with a browser extension
https://github.com/internetarchive/wayback-machine-webextension
>>
MEDIUM priority (Important overall)

3. Seed torrents for as long as possible, rare data forever. Make sure to look up a guide for your router to PORT FORWARD your torrent client port, to substantially increase your upload (and your download) speed. In low population torrent swarms, if no one is port forwarded then you might not be able to connect to each other at all and exchange any data despite having it.
Requirements: As much or as little bandwitdh you want (you can set the limits if you need to)
https://github.com/qbittorrent/qBittorrent (Recommended client, especially to replace uTorrent)

4. Archive web pages you want to have a local copy of with a "Web Extension for saving a faithful copy of a complete web page in a single HTML file with a single click"
https://github.com/gildas-lormeau/SingleFile

5. Archive videos with "GUI front-end for youtube-dl, yt-dlp and other compatible video downloaders"
https://github.com/axcore/tartube

6. "Capture or record any area of your screen and share it with a single press of a key"
https://github.com/ShareX/ShareX

7. Archive entire websites you want to have a local copy of
https://www.httrack.com/
>>
8. Publish the data that you have archived that isn't easily or at all available online. You can easily create torrents yourself in your torrent client and then share the magnet link to it anywhere online for anyone to access and, as long as DHT (Distributed Hash Table, decentralized way to share torrents without the need for any specific tracker) is enabled in settings (on by default), your files will be searchable on DHT by DHT crawlers, local or online (for example https://btdig.com/, where you can actually also search for FILE NAMES within all DHT torrents)
(archive.org also creates torrents for all uploads automatically but their torrents shouldn't be relied on because of an error-prone implementation and since they can also break when more files are uploaded or if the item's metadata changes, which includes even getting a new comment on the item)


OTHER useful things:

- In your torrent client settings add the best trackers to be automatically added for all of your newly added torrents (helps more easily connect to peers, especially in obscure torrents)
https://github.com/ngosang/trackerslist

- If you have a torrent in your torrent client which is stuck at downloading metadata, you can see if the .torrent file containing the metadata was cached online. Copy the v1 torrent hash of your torrent and replace it in the link below which uses itorrents.net and btcache.me services to check:
https://downloadtorrentfile.com/hash/c8295ce630f2064f08440db1534e4992cfe4862a

- Look into running a node for I2P (anonymous private network within the global internet)
Requirements: Mostly bandwidth, more info: https://geti2p.net/en/faq
https://geti2p.net/

- Look into running Tor/Hyphanet(Freenet)/IPFS/YaCy/SearXNG nodes

- Easily capture and digitize all data AND METADATA from optical media (CDs, DVDs, Blu-rays...) with Media Preservation Frontend (MPF)
https://github.com/SabreTools/MPF
>>
- "A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI"
https://github.com/bitmagnet-io/bitmagnet

- "ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline"
https://github.com/ArchiveBox/ArchiveBox

- Look into donating your PC resources to be used more intensively in projects:
BOINC (Berkeley Open Infrastructure for Network Computing): https://boinc.berkeley.edu/projects.php


- Additional archiving tools: https://github.com/iipc/awesome-web-archiving

- Additional links to archiving and similar communities:
https://wiki.archiveteam.org/index.php/Archiveteam:IRC
https://www.reddit.com/r/Archiveteam
https://www.reddit.com/r/DataHoarder
https://www.reddit.com/r/DataHoarder/wiki/index/ - Hardware and software for data hoarding FAQ
https://www.reddit.com/r/lostmedia
https://www.reddit.com/r/GamePreservationists
https://www.reddit.com/r/torrents
https://www.reddit.com/r/qBittorrent
https://annas-archive.se/torrents
>>>/t/

What are you archiving or want to archive?
Do you have or know anyone who has some rare interesting data or media not available online?
>>
Added last:
>- If you have a torrent in your torrent client which is stuck at downloading metadata, you can see if the .torrent file containing the metadata was cached online. Copy the v1 torrent hash of your torrent and replace it in the link below which uses itorrents.net and btcache.me services to check:
https://downloadtorrentfile.com/hash/c8295ce630f2064f08440db1534e4992cfe4862a
>>
>>110206580
Can someone get these fuckers to finally merge tmpfs support for their container image? No, I won't use some random fork or maintain one myself.
https://github.com/ArchiveTeam/warrior-dockerfile/pull/90
I uploaded a couple TBs but stopped once storage prices got retarded. I have 8 residential IPs just begging to be used.
>>
>>110206730
Make a ramdisk and put the software there, if it just works on windows it must be easy to set up on linux too
>>
So what’s the easiest way to download videos from ThisVid?
>>
>>110206931
yt-dlp
>>
tldr
>>
>>110207318
http://warrior.archiveteam.org/
>>
>>110207119
>>110206931
How do i get yt-dlp to add the author/uploader etc to the filename if %(uploader)s, %(channel)s, etc don't work?

Is there a way I can manually tell yt-dlp, "hey, this bit of html/page source/whatever is where the text field in question is"?

Same for if it's not detecting the video to rip at all but I can find the mp4 url in the page source/elements?
>>
I need a portable HDD to plug into my PC's via USB, but everything is overpriced right now and I want to buy a refurbished or used internal HDD instead and just use a USB enclosure

However, the only drives I can find for a really good price are SAS rather then SATA, and I can't seem to find a enclosure that will work with SAS drives that would function like a portable drive that's not obscenely overpriced, unless I'm just not looking in the right places.

Do they not exist or what?
>>
>>110207640
The easiest way is to ask any online free AI to read the docs and give you the command or the steps to change the config.

If some things are not possible then the AI can create a script to scrape the web and rename the files at the end.

For sites where yt-dlp doesn't work, the easiest thing to try is Video Download Helper extension.
>>
I am still looking for altchan archives including the ones that can't be mentioned.
>>
>>110207722
I don't use AI tools, like, at all.

So I wouldn't even know how to find one to ask it this stuff. And I'm not giving an AI company my email address and phone number to sign up (the one time I tried to sign up with ChatGPT, it required both, so I couldn't just use a throwaway email service)

>>110207875
I don't think you get banned or have your posts deleted for mentioning 8ch anymore, I haven't seen that happen in many, many years. Anyways I don't know of any archives for any other imageboards, sorry anon
>>
Not sure where else to ask this:

Why do so many albums on bunkr have less (or even zero) images or videos when you open them compared to the amount of files it said they had in the search results?

There's some albums that in the search results were marked as containing hundreds of images that then only have a few when I click the album to view the files, and tons of them straight up just have no files after opening them even if it said otherwise in the search results

Is this from DMCA takedowns or something? Is there any way to view the original set of files a album had just in terms of filenames even if the video/image isn't viewable?
>>
>>110207640
>>110207884
You could try modifying the extractor script, it's Python.
>>
>>110207884
You should be able to ask ChatGPT a few questions without registering, if you don't have a single burner gmail then get into local models https://owo.vg/g/lmg
>>
>>110207935
If you have a guide on how i'd do that, I'll give it a look, but the extent of my coding/programming/scripting knowledge is occasionally stumbling my way through using command line tools like this to begin with, and even then I struggle with them
>>
>>110207888
bunkr servers are down a lot
or maybe indexing wasnt updated
>>
File: 127337671_p0.jpg (340 KB, 850x850)
340 KB JPG
>>110207946
There's no way of doing it through the CLI.
I don't have a guide or specialized experience for writing extractors, it would also depends on the website I suppose, sorry.
Python is worth learning in general is you're doing scripting often though, it's way better than shell.
>>
>>110206580
>archiveteam.org
Why are these faggots so incompetent?
They seems to archive (badly) the most irrelevant shit on the internet while all the good stuff is routinely lost, are they a psyop?
>>
>>110208075
You're overestimating my knowledge, I don't even know what "shell" is
>>
>>110207875
>>110207884
https://archivohot.net/ seems to be an archive for a altchan
>>
>>110208479
also this? https://arhivach.vc
>>
>>110208265
>while all the good stuff is routinely lost
like?
>>
>>110210963
Porn?
>>
>>110211846
private trackers should have enough
>>
>>110213924
Private trackers are not public archive, in fact they will be one of the reason why media will be lost, all because some faggots want to feel like they have authority and powertrip
>>
>>110207655
Well, I found this:

https://www.amazon.com/OSENTRiX-GX200S-External-Enclosure-Driveless/dp/B0HLR9JDZS

Anybody familiar with this brand?
BTC
bc1qtm4rep7kyp2p28hlkfktpnd0j7uew47vnt2mev
LTC
ltc1q9xlaysse0mpfwf025esd69kt9zu3md22gh23du
ETH
0x6abA18D203520c8F90ca63eE7426a7bC345D98e6
SOL
E5ekoiauCR8yiNp451dmuPzhUxo84P5x3m9WJBK5kxZe
XRP
r3jYXNdMk2Vz5hXhKCKE9re8uz93JQFG1r
DOGE
DSBq5jRWkiwmEh1G7zBwmQDgRkvRr61fmb
XMR
85CTwq5wGjWZw5GgzgwZPrHyZxpLXMK5oZSSSk5j4x9w1bpqzisaWAUZYaAPfh6rP2MZJcaFqF1XmEzkw6YCqJSS2Txu6Z8




Delete Post: [File Only] Style:
[Disable Mobile View / Use Desktop Site]

[Enable Mobile View / Use Mobile Site]

All trademarks and copyrights on this page are owned by their respective parties. Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.