{"id":366,"date":"2025-02-05T23:27:32","date_gmt":"2025-02-06T04:27:32","guid":{"rendered":"https:\/\/dpm.darkhorselinux.org\/?page_id=366"},"modified":"2025-02-14T01:10:55","modified_gmt":"2025-02-14T06:10:55","slug":"dpm-repository-format","status":"publish","type":"page","link":"https:\/\/dpm.darkhorselinux.org\/?page_id=366","title":{"rendered":"DPM Repository Format"},"content":{"rendered":"\n<p>The DPM repository consists of metadata and packages.  The metadata is used to generate a small file-based database that is downloaded, cached, and used by <code>DON<\/code> for repository operations.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Repository Caching<\/h2>\n\n\n\n<p>Ultimately, what DON downloads and caches from the repository to do what it does, is a file at <code>${repository_root}\/metadata.db<\/code>. <\/p>\n\n\n\n<p>In order to generate metadata.db, the repo creation tool, <code>dpm_create-repo<\/code> first generates a hierarchal tree structure. This is a sample repository tree used to generate metadata.db:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>repository\/\n\u251c\u2500\u2500 metadata.db\n\u251c\u2500\u2500 metadata\/\n...\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 4ffd74ecd2df9fb9748cc8ca91fcc8fc71323380e062b4f3...19226\/\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 metadata\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 ARCHITECTURE\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 AUTHOR\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 CONTENTS_MANIFEST_DIGEST\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 DEPENDENCIES\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 DESCRIPTION\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 LICENSE\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 NAME\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 PACKAGE_DIGEST\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 PROVIDES\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 REPLACES\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 SOURCE\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 VERSION\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 signatures\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0     \u251c\u2500\u2500 contents.signature\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0     \u251c\u2500\u2500 hooks.signature\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0     \u2514\u2500\u2500 metadata.signature\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 56213bb6cec8932c7adff33a97c2dfe7d3758c32dc46effb...e239a\/\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 metadata\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 ARCHITECTURE\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 AUTHOR\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 CONTENTS_MANIFEST_DIGEST\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 DEPENDENCIES\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 DESCRIPTION\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 LICENSE\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 NAME\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 PACKAGE_DIGEST\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 PROVIDES\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 REPLACES\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u251c\u2500\u2500 SOURCE\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 VERSION\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 signatures\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0     \u251c\u2500\u2500 contents.signature\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0     \u251c\u2500\u2500 hooks.signature\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0     \u2514\u2500\u2500 metadata.signature\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 REPO_MANIFEST\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 pubkeys\n\u2502\u00a0\u00a0  \u00a0\u00a0 \u2514\u2500\u2500 DHL2.pubkey\n\u2514\u2500\u2500 packages\n    \u251c\u2500\u2500 aarch64\n    \u2502\u00a0\u00a0 \u251c\u2500\u2500 myapp-0.1.2-0.dhl2.aarch64.dpm\n    \u2502\u00a0\u00a0 \u2514\u2500\u2500 myapp-selinux-0.1.2-0.dhl2.aarch64.dpm\n    \u251c\u2500\u2500 noarch\n    \u2502\u00a0\u00a0 \u251c\u2500\u2500 myapp-0.1.2-0.dhl2.noarch.dpm\n    \u2502\u00a0\u00a0 \u2514\u2500\u2500 myapp-selinux-0.1.2-0.dhl2.noarch.dpm\n    \u2514\u2500\u2500 x86_64\n        \u251c\u2500\u2500 myapp-0.1.2-0.dhl2.x86_64.dpm\n        \u2514\u2500\u2500 myapp-selinux-0.1.2-0.dhl2.x86_64.dpm\n<\/code><\/pre>\n\n\n\n<p>It can be simplified as:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>repository\/\n\u251c\u2500\u2500 metadata\/\n\u2502   \u251c\u2500\u2500 &#91;package_digest_hash]\/\n\u2502   \u2502   \u251c\u2500\u2500 metadata\/\n\u2502   \u2502   \u2502   \u2514\u2500\u2500 &#91;metadata files]\n\u2502   \u2502   \u2514\u2500\u2500 signatures\/\n\u2502   \u2502       \u2514\u2500\u2500 &#91;signature files]\n\u2502   \u251c\u2500\u2500 REPO_MANIFEST\n\u2502   \u251c\u2500\u2500 pubkeys\/\n\u2502   \u2502   \u2514\u2500\u2500 &#91;public signing keys]\n\u2502   \u2514\u2500\u2500 metadata.db\n\u2514\u2500\u2500 packages\/\n    \u251c\u2500\u2500 aarch64\/\n    \u2502   \u2514\u2500\u2500 &#91;architecture-specific packages]\n    \u251c\u2500\u2500 noarch\/\n    \u2502   \u2514\u2500\u2500 &#91;architecture-independent packages]\n    \u2514\u2500\u2500 x86_64\/\n        \u2514\u2500\u2500 &#91;architecture-specific packages]<\/code><\/pre>\n\n\n\n<p>At the top level, you will see a <code>metadata<\/code> directory and a <code>packages<\/code> directory.  You will also see a metadata.db and REPO_MANIFEST at the top level.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Metadata Directory<\/h2>\n\n\n\n<p>The metadata tree in the repository structure closely resembles the <code>DPM Backing Tree<\/code>, and this is by design to facilitate what <code>DON<\/code> will do with them. <\/p>\n\n\n\n<p>Once the DPM Repo metadata is created, it then populates the small database at <code>metadata.db<\/code> from that data.<\/p>\n\n\n\n<p>The <code>DON<\/code> client downloads the repository metadata db, and caches this repository metadata locally with a configurable <code>TTL<\/code> (Time-to-Live). Once this cache expires, <code>DON<\/code> refreshes this cache from that repository at the time of an interaction (or unless the command to refresh it is run).<\/p>\n\n\n\n<p>This metadata cache is for the purpose of listing package dependencies and file\/package lookups so that the data to determine which package provides a file, or what files are provided by a package, is present in addition to what packages are required to be installed for any package in the repository to be installed.<\/p>\n\n\n\n<p>There are a few subtle differences in the repository metadata from the DPM Backing Tree:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Hooks are not copied<\/li>\n\n\n\n<li>A file called <code>REPO_MANIFEST<\/code> exists, generated by the <code>dpm_create_repo<\/code> tool at the time of the last update of the repository.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">The REPO_MANIFEST File<\/h3>\n\n\n\n<p>The <code>REPO_MANIFEST<\/code> file is generated when the repository metadata is generated. During repository creation the utility finds all .dpm files recurvisely in the <code>packages<\/code> top-level directory and then calculates the cryptographic checksum to create a two column line delimited table file of the following format:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>$CHECKSUM $file_path\\n<\/code><\/pre>\n\n\n\n<p>This pairing of repository paths to package digest hashes compliments the pairing of package digest hashes to package metadata in the generated metadata tree just discussed.<\/p>\n\n\n\n<p>Any file or directory that does not end in .dpm is skipped during the generation of the REPO_MANIFEST file.<\/p>\n\n\n\n<p>The $file_path is relative to the top-level directory of the repository.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">PUBKEYS<\/h3>\n\n\n\n<p>This directory contains the public keys and everything needed to validate package cryptographic signatures.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Packages Directory<\/h2>\n\n\n\n<p>The packages directory in the DPM repository is very simple.  It is an arbitrary directory structure containing at least .dpm files.<\/p>\n\n\n\n<p>If files that do not end in .dpm are included this is fine and these files will not be processed during a run of <code>dpm_create-repo<\/code>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Repository Creation<\/h2>\n\n\n\n<p>After building out the packages directory structure, the user runs <code>dpm_create-repo<\/code>. This command generates the repository metadata tree and populates the <code>REPO_MANIFEST<\/code> inside it, before using that data to create <code>metadata.db<\/code>.<\/p>\n\n\n\n<p><code>dpm_create-repo<\/code> walks through the packages directory tree, extracting the package metadata, signatures archives to do this before rendering the downloadable database.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Repository Usage<\/h2>\n\n\n\n<p><code>DON<\/code> downloads cache repository metadata with a configurable TTL. It will then query the databases it has cached from all repositories when performing an operation.<\/p>\n<div class=\"pdfprnt-buttons pdfprnt-buttons-page pdfprnt-bottom-right\"><a href=\"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=%2Fwp%2Fv2%2Fpages%2F366&print=pdf\" class=\"pdfprnt-button pdfprnt-button-pdf\" target=\"_blank\" ><span class=\"pdfprnt-button-title pdfprnt-button-pdf-title\">[ EXPORT TO PDF ]<\/span><\/a><a href=\"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=%2Fwp%2Fv2%2Fpages%2F366&print=print\" class=\"pdfprnt-button pdfprnt-button-print\" target=\"_blank\" ><\/a><\/div>","protected":false},"excerpt":{"rendered":"<p>The DPM repository consists of metadata and packages. The metadata is used to generate a small file-based database that is downloaded, cached, and used by DON for repository operations. Repository Caching Ultimately, what DON downloads and caches from the repository to do what it does, is a file at ${repository_root}\/metadata.db. In order to generate metadata.db, &hellip; <a href=\"https:\/\/dpm.darkhorselinux.org\/?page_id=366\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">DPM Repository Format<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-366","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=\/wp\/v2\/pages\/366","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=366"}],"version-history":[{"count":37,"href":"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=\/wp\/v2\/pages\/366\/revisions"}],"predecessor-version":[{"id":511,"href":"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=\/wp\/v2\/pages\/366\/revisions\/511"}],"wp:attachment":[{"href":"https:\/\/dpm.darkhorselinux.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=366"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}