Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
-
RE: https://glammr.us/@overholt/117049333496122205
Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
The conclusion we came to was that there was •no• storage medium that met their long-term preservation goals — true then, and almost certainly still true today. The best approach is perpetual and redundant forward copying of the data, from one physical storage device to another.
An archive isn’t a physical thing; it's a process.
@inthehands LOCKSS (Lots of Copies Keep Stuff Safe) tries to mitigate the problem of integrity when copying files forward. TL:DR - min 4 copies with continuous file integrity checks.
(edit: it's mentioned in the Gizmodo article)
-
RE: https://glammr.us/@overholt/117049333496122205
Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
The conclusion we came to was that there was •no• storage medium that met their long-term preservation goals — true then, and almost certainly still true today. The best approach is perpetual and redundant forward copying of the data, from one physical storage device to another.
An archive isn’t a physical thing; it's a process.
@inthehands data storage is the easy part. Being able to decode the stored data long after the computers that wrote it are obsolete and no machine is available that can run the old binaries is the fun part. Even if you have carefully forward copied all the bits onto media you can read, the bits are just the starting point.
-
@gbargoud @inthehands But isn't the fact that it's in your home another potential failure mode? A house fire can undo all your careful rebalancing in an hour. (Of course, you don't mention offsite storage; you could have a copy at your sister's or something.)
Yes that is true but that's the same failure mode of every single site storage system, including libraries as in the original quoted post. Having additional redundancy in multiple locations is important for some information but each location needs to have that kind of maintenance happening for properly long term storage
-
RE: https://glammr.us/@overholt/117049333496122205
Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
The conclusion we came to was that there was •no• storage medium that met their long-term preservation goals — true then, and almost certainly still true today. The best approach is perpetual and redundant forward copying of the data, from one physical storage device to another.
An archive isn’t a physical thing; it's a process.
@inthehands Related: https://www.lockss.org/
-
RE: https://glammr.us/@overholt/117049333496122205
Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
The conclusion we came to was that there was •no• storage medium that met their long-term preservation goals — true then, and almost certainly still true today. The best approach is perpetual and redundant forward copying of the data, from one physical storage device to another.
An archive isn’t a physical thing; it's a process.
@inthehands the quipy way I've heard this described is that in archival efforts, you dont have "storage," you have "movage."
-
RE: https://glammr.us/@overholt/117049333496122205
Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
The conclusion we came to was that there was •no• storage medium that met their long-term preservation goals — true then, and almost certainly still true today. The best approach is perpetual and redundant forward copying of the data, from one physical storage device to another.
An archive isn’t a physical thing; it's a process.
My preservation strategy is, no joke, a Nextcloud server that backs itself up on top of the data it contains - then pushes it to my desktop and laptop and partly to my smartphone. Bam, triple redundancy -
RE: https://glammr.us/@overholt/117049333496122205
Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
The conclusion we came to was that there was •no• storage medium that met their long-term preservation goals — true then, and almost certainly still true today. The best approach is perpetual and redundant forward copying of the data, from one physical storage device to another.
An archive isn’t a physical thing; it's a process.
@inthehands I've worked in numerical banks that archive their records. They generally use high quality optical storage. Gold or silver based cdr or mdisk. Both thought to last 30-100 years
-
RE: https://glammr.us/@overholt/117049333496122205
Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
The conclusion we came to was that there was •no• storage medium that met their long-term preservation goals — true then, and almost certainly still true today. The best approach is perpetual and redundant forward copying of the data, from one physical storage device to another.
An archive isn’t a physical thing; it's a process.
@inthehands
That's totally true. I watched a French documentary on long term nuclear waste storage sites which have to keep them for many centuries (millenia even!) and they were researching solutions to keep instructions on how to (not) handle and keep away from the site for future generations (civilisations even) who won't even speak speak the same language. They chose acid free paper in climate controlled rooms for the first decades and century. They were also speaking about genetically engineering a new bread of cats to glow near nuclear material and had some idea on songs about being fearful of glowing cats because it meant danger. They say that's how stories travel through millenias and that could help future people to not access the dangerous zones of the site, even after a civilisation collapse. Far fetched but interesting. -
@inthehands I've worked in numerical banks that archive their records. They generally use high quality optical storage. Gold or silver based cdr or mdisk. Both thought to last 30-100 years
@Eurospoofer @inthehands You still have to solve the availability of reading hardware and software in 100 years.
-
@inthehands @overholt Yup, I’ve been a “Digital Archivist” for 30 years, personally and professionally. Only the process will save you. All media fails.
@JustinDerrick definitly so. @inthehands @overholt
-
We need the equivalent of stone tablets.
3D-printing using ECCs onto some kind of ceramic alloy that's super hard and super temperature resistant.@megatronicthronbanks @inthehands The trouble with extremely durable high-density media is that reading them still requires sophisticated machines. In 100 years time, you probably won't find one that works, nor instructions for building one.
-
@inthehands My very non-technical historian husband says you should have multiple types of media and multiple locations.
@ClimateJenny @inthehands On different tectonic plates.
-
@megatronicthronbanks @inthehands The trouble with extremely durable high-density media is that reading them still requires sophisticated machines. In 100 years time, you probably won't find one that works, nor instructions for building one.
@mansr @inthehands What if it's just barcodes?
-
@mansr @inthehands What if it's just barcodes?
@megatronicthronbanks @inthehands How are you going to store the instructions for decoding them?
-
@megatronicthronbanks @inthehands How are you going to store the instructions for decoding them?
@mansr @inthehands Why would you need to? You only need to describe the format.
It's not a DVD or DLT, that requires precise physics to decode.
-
@mansr @inthehands Why would you need to? You only need to describe the format.
It's not a DVD or DLT, that requires precise physics to decode.
@megatronicthronbanks @inthehands So where will you store the description then? You can't expect anyone to just look at a bar/2D code and infer the arithmetic required to decode it.
-
@megatronicthronbanks @inthehands So where will you store the description then? You can't expect anyone to just look at a bar/2D code and infer the arithmetic required to decode it.
@mansr @inthehands In a few human languages, on pages scattered throughout to protect them. With diagrams.
We did it with a gold record, and that was for aliens.
-
@mansr @inthehands In a few human languages, on pages scattered throughout to protect them. With diagrams.
We did it with a gold record, and that was for aliens.
@megatronicthronbanks @inthehands We have yet to receive confirmation that anyone has successfully decoded that gold record.
Fair enough, though, you could engrave a decoding manual in human languages, although it would be more than a few pages (a few hundred more likely if you want to be thorough), and you'd want translations to cover at least the major language families of today.
-
RE: https://glammr.us/@overholt/117049333496122205
Back in 1999 or so, I was on the team that helped Minnesota Public Radio choose a long-term storage format for their (extensive and valuable!!) audio archives, which they were about to digitize.
The conclusion we came to was that there was •no• storage medium that met their long-term preservation goals — true then, and almost certainly still true today. The best approach is perpetual and redundant forward copying of the data, from one physical storage device to another.
An archive isn’t a physical thing; it's a process.
I managed the computers for my dad’s video store in the late twentieth and early 21st centuries (1995 - 2007).
The store’s rental software was based around a MS SQL server in RAID 1 and I replaced both drives annually just to be safe. Even then, I ended up having to do an emergency recovery because I moved to another country and let the replacement schedule slip.
So the only way to store data long-term is to re-establish the Knights Templar and task them with guarding it.
-
@Eurospoofer @inthehands You still have to solve the availability of reading hardware and software in 100 years.
@jernej__s @Eurospoofer @inthehands
This is already a problem with proprietary systems where the vendor goes out of business but the data has to be retained and usable for at least 7 years for legal reasons.
I worked at a bank in this situation and it was a giant pain because the system couldn’t run on any modern / approved operating system so it couldn’t be hosted in any location without massive security issues and mitigations.
So the first step is open data formats that can be maintained
