View source for file-storage

You do not have permission to edit this page, for the following reason:
The action you have requested is limited to users in the group: Users.
You can view and copy the source of this page.
{{stub}}

'''<dfn>File Storage</dfn>''' is the practice of storing main (primary) site content in flat files in a filesystem.

== Why ==
Many indieweb [[projects]] are storing their data in flat files on the file system for reasons of simplicity and robustness.  

See also the [[databases-antipattern]] for reasons why flat files for storage are growing in popularity compared to databases.

== How ==
These flat files typically are either:
* [[Markdown]] with custom (per project) extensions for various metadata
* HTML+[[microformats]] (e.g. [[hAtom]] and/or [[h-entry]])

And contain either:
* one post per flat file
* time sharded: a set of posts per fixed time period per flat file

== IndieWeb Examples ==
IndieWeb community members that use file storage on their personal site.

=== Tantek ===
{{t}} on tantek.com uses flat files for primary storage of content:
* [http://tantek.com/2010/001/t1/declaring-independence-building-it 2010-01-01]... present: [[Falcon]] which stores data in HTML+microformats flat files, 6 per year (1 per [[bim]]).  See '''[[Falcon#Storage_format|Falcon storage format]]''' for details.
* [http://tantek.com/log/2002/08.html 2002-08]...[http://tantek.com/log/2008/08.html 2008-08] static site directly served from primary storage of flat files, up to 12 per year (1 per Gregorian month).

=== Aaron Parecki ===
{{main|p3k#Storage}}
{{aaronpk}} on [http://aaronparecki.com aaronparecki.com] uses [[p3k]] - one file per post, YAML extensions for meta info - since ~2011. See '''[[p3k#Storage|p3k Storage details]]'''.

=== Kevin Marks ===
{{kevinmarks}} on [http://kevinmarks.com kevinmarks.com] stores posts formatted as HTML in [https://github.com/kevinmarks/kevin-marks.com GitHub] and deploys statically to Heroku.

=== Bear ===
[[User:Bear.im|Bear]] at https://bear.im/ uses [[Hakkan]] uses flat files for storage of all content:
* posts - since 2003-12-18 stored as text with some textual/markdown/whatever markup, with one file per post, named year+day-of-year, key:value header for metadata and a json blob for connected resources
* webmentions - stored as html in the same directory as the post
* comments - stored as html in the same directory as the post
* feeds and related

=== Bret C. ===
{{bret}} uses the following on bret.io:
* Stores site content in git
* Renders to html using [[jekyll]]/gh-pages
Yaml headers (aka front-matter) with [[GitHub]] flavored [[Markdown]] mixed with liquid template include tags for post attachment metadata.  Yuuuuuck!  This is what jekyll sets you up with.  It has its ups and downs, but its a mix of 3 weird languages which really bothers me.

=== Rascul ===
{{rascul}} stores content for https://rascul.io in a [https://git.rascul.io/crash/content git repository] and uses [[crash]].

=== Barnaby Walters ===
Stores all canonical copies of [http://waterpigs.co.uk waterpigs.co.uk] site data in the filesystem and uses [[Taproot]] for publishing.

* notes and articles as YAML files with HTML in (wants to change these to HTML+mf2 for browsability, standards support)
* timestamped HTML archives (along with HTTP headers in *-headers.txt) in URL-mimicking folder structures
* Sheet music in ABC files

=== Mark87 ===
Testing at [http://markmurfin.com/editor.php Editor] and [http://markmurfin.com/viewer.php Viewer]
* Simple, working Editor saves simple articles as html files to the filesystem.
* Viewer system loads the html files and can output them with different markup, css, adding h-data, etc.
* Wants to save html files with microdata schema for super cool idea that I have

=== Emma K. ===
[[User:notenoughneon.com|Emma]] at http://notenoughneon.com using [[Neonblog]]. Posts are stored in html+mf2 files. The feed is dynamically generated from parsed microformats with an index persisted in JSON.

=== Kartik Prabhu ===
2017-05-02: {{kartik}} at https://kartikprabhu.com. 
* [[Notes]] are stored as html+mf2 and [[JSON]] files, with a copy in the [[database]] (only create and update currently). Posts are currently generated from the database (to be changed soon...).
* [[Articles]] content is stored as Django templates wit meta-data in database.

=== Dmitri Shuralyov ===
{{ds}} uses filesystem-backed implementations of services to store all dynamic content of https://dmitri.shuralyov.com. It feels cleaner, simpler, more lightweight than pulling in a heavy database dependency, and sufficient for current needs.

Specifically, blog posts (and their comments), issues (within issue trackers), events, notifications, reactions and users are stored and modified as flat files on disk. The functionality is implemented in [https://dmitri.shuralyov.com/packages/.../fs various Go packages named <code>fs</code>].

=== capjamesg === 

{{capjamesg}} on [https://jamesg.blog jamesg.blog] stores his posts, coffee reviews, and projects in markdown. Each markdown file makes use of YAML for metadata, such as post titles and review dates.

=== ... add yourself ... ===

=== ... many more [[Jekyll]] users ===

=== ... many more [[Hugo]] users ===

=== Want to use file storage ===
IndieWeb members that *want* to use File Storage on their personal site:

* add yourself.

== Other Independent Examples ==
=== David Baron ===
David Baron on http://dbaron.org/ uses flat HTML files, one per blog post, one for his blog index page, since 2002.

== Intranet Examples ==
=== XOXCO ===
XOXCO uses flat [[markdown]] files as the storage for their internal wiki which they use and contribute to from [[Slack]]
* <span class="h-cite"><time class="dt-published">2015-05-21</time> <span class="h-card p-author">Ben Brown</span>: <cite class="p-name">[https://medium.com/why-not/our-slack-wiki-e4f6d23a3495#3aad Our Slack Wiki: No Database]</cite>, <span class="u-url">https://medium.com/why-not/our-slack-wiki-e4f6d23a3495#3aad</span></span>
** http://aaronparecki.com/bookmarks/2015/05/21/3/wiki-slack-database

== Flat file "CMS" systems in the wild ==
* [[Grav]] is a PHP, YAML, Twig and Markdown Extra based storage project which looks promising.
* [[Kirby]]
* [[Statamic]]
* [[Blot]]
* [[Jekyll]]
* [[Hugo]]
* [[Eleventy]]

== Software ==
* [[QuartzDB]]

== Criticism ==
=== Command Line Configuration ===
The process of uploading and setting up flat file storage on shared [[hosting]] may require command line configuration such as:
* <kbd>chmod -R 0777</kbd>[http://indiewebcamp.com/irc/2014-10-16/line/1413497857282]
Any flat file storage solution should seek to minimize and eliminate any command line based configuration needs.

== See Also ==
* [[storage]]
* [[database]]
* [[database-antipattern]]
* [[database-cache]]
* [[projects]]
Return to file-storage.
Retrieved from "https://indieweb.org/file-storage"