Writing an app with SvelteKit

This blog post is going to focus on how I used SvelteKit to solve a practical problem by writing a small app with SvelteKit and Svelte.

For some background, I’ve taken up Minecraft modding in my free time. Since it’s Java-based it’s easy to give it a try if you’re already familiar with the language. Because I didn’t want to come up with a complete mod on my own, I decided to just do an addon to an already existing, quite popular mod: Ars Nouveau (AN).

The Minecraft modding community has a couple of solutions for documenting mods. One of these solutions is Patchouli, an API that allows the documentation of a mod in the form of an in-game book. Both Ars Nouveau and my own mod use it to provide our documentation.

The problem

In-game documentation is all well and good, but it’s not always available to curiously minded players. There’s always someone asking for an online wiki to access information on the go or when they are currently unable to play.

Maintaining such a wiki is quite a lot of work though, especially when internationalization is involved. Also, it feels redundant, since all the information is already there in the in-game documentation, isn’t it?

The idea

Quite quickly, the idea for a wiki page that accesses the Patchouli resources and displays them in a user readable format was born.

Since I recently had a bit of free time and wanted to try my hands at a real project with SvelteKit, I decided to give it a try.

The constraints and requirements

The wiki idea came with a couple of constraints and requirements:

• Always up-to-date, no user or admin-input required
• Should support all the languages supported in Ars Nouveau
• Should work on both mobile and desktop
• Should not require direct access to the AN Github repository or any input of the AN author

Especially the requirement that the wiki should automatically display the newest content, without having any direct tie-ins to the Ars Nouveau repository was tricky.

The easiest way to do it would have been to add a Github Action to the Ars Nouveau repository and automatically build & deploy the Wiki app with each release, but since I don’t have rights to add an Action to the repo, I’d have to use another way.

Luckily, Github also has a REST-API that allows access to repository contents, with no access restrictions for public repositories.

The implementation

With a solution to the first and biggest problem, work on implementing the wiki could start.

Quite soon I would notice a problem though: The Github API imposes a rate limit of 50 calls per hour on unauthenticated users. This is raised to 500 calls per hour for authenticated users.

Just the .png-files to be able to display the items added by the mod are about 250 files, each needing a call of their own. Each crafting recipe to acquire the item in game is also a file of its own and so are the Patchouli documentation pages.

It’s therefore impossible to acquire all the information the wiki needs with the Github repository API. Is it really, though?

Setting up the project

Before we can try to make the impossible possible, the SvelteKit project needs to be created.

npm create svelte@latest wiki

Agree to the installation of the required packages and choose Skeleton project when asked which type of project should be set up.
I’ve chosen to use type checking with TypeScript syntax, to include ESLint, Prettier and Playwright.

The project can be started by entering the directory:

cd wiki
npm run dev

Making the impossible possible

With the setup done, we can start implementing the actual logic. It might be impossible to acquire the necessary files directly through the API, but the documentation also mentions the possibility to get the whole repository as a zipball!

Since all the content the app should display is contained in the Github repository, it needs to be fetched before the page can be displayed.

In the previous post about SvelteKit, I’ve explained how one can use the +layout.ts file to execute code before the page is displayed. This is where the loading of wiki files should happen, so we create a new +layout.ts file in the src/routes directory.

import { browser } from $app/environment;

export async function load({ fetch }: LoadEvent) {
   const url = browser ? 'https://api.github.com/repos/baileyholl/Ars-Nouveau/zipball' : '/api/zipball';
   fetch(url).then(function (response) {
      if (response.status === 200 || response.status === 0) {
         return response.blob();
      } else {
         console.log(response.statusText);
         return Promise.reject(new Error(response.statusText));
      }
   })
   .then(() => {/*process zip ball*/}
}


		

To make sure that the server does not get rejected by the rate limit when we access the Github API we check if the code is running in a browser or on the server and make an authorized call if it is running on the server.

If it’s running in the browser, the API is called directly. If not, it is sent to an API route that we still need to create: /api/zipball.

To do that, we create the api and zipball directories and create a new file in the zipball directory: src/routes/api/zipball/+server.ts

import { env } from '$env/dynamic/private';
import { apiBaseURL } from '$lib/utils/apiUtils';

const headers = new Headers({
   Accept: 'application/vnd.github+json',
   Authorization: `Bearer: ${env.GITHUB_API_KEY}`
});
const fetchParameters = { method: 'GET', headers };

export function GET() {
   return fetch(apiBaseURL(), fetchParameters).catch((reason) =>
      console.log(JSON.stringify(reason))
   );
}


		

By importing the private dynamic environment, SvelteKit can access an environment variable called GITHUB_API_KEY and authorize the call from the server to Github, without exposing the API key to end users.

The environment variable needs to be configured in the local environment and later in the deployed environment as well.

The different load URLs for client and server needed to be implemented, since the future host (see Part 3) of the app only permits 4.5mb big responses from serverless functions and with the repository zip being 4.9mb large it needs to be fetched directly from the API on the client side.

Curses, there’s CORS

When trying to run the above code, it will throw an error while trying to execute in the browser since the Github API does not include a CORS header, which leads to the browser automatically closing the request with an error to protect the user.

Without going into to much detail, this can be circumvented locally by running a cors-anywhere proxy. Yasinuslu on github has a finished docker image ready for local use which is what I’ll be using for local development.

Cors-anywhere has a couple of guides available on how to set up a deployed proxy while making sure that it can’t be abused by others.

import { dev } from '$app/environment';

export const apiBaseURL = () => {
   if (dev) {
      return 'http://localhost:30080/https://api.github.com/repos/baileyholl/Ars-Nouveau/zipball';
   } else {
      return '<your deployed cors proxy>/https://api.github.com/repos/baileyholl/Ars-Nouveau/zipball';
   }
};

With a new utility function that checks if the application is running in dev mode or deployed mode, requests can be directed to the correct CORS proxy.
// in +layout.ts
const url = browser ? apiBaseURL() : '/api/zipball';

// In +server.ts
export function GET() {
   return fetch(apiBaseURL(), fetchParameters).catch((reason) =>
      console.log(JSON.stringify(reason))
   );
}


		

Parsing the zipball

To parse the zipball JSZip is used. It is a great library that works both in a Node.js and browser environment which is what we need for SvelteKit. Install it using

npm install jszip

To make sure it can work in both environments, we’ll need to convert the Blob received from the fetch-request into an ArrayBuffer:

import JSZip from 'jszip';

return fetch(url)
   .then(function (response) {
      if (response.status === 200 || response.status === 0) {
         return response.blob();
      } else {
         console.log(response.statusText);
         return Promise.reject(new Error(response.statusText));
      }
   })
   .then((blob) => blob.arrayBuffer())
   .then((blob) => {
      return JSZip.loadAsync(blob);
   })

After the zip has been loaded, the necessary data can be extracted. To make sure that it is available in following requests and while navigating through the app when it has been hydrated on client side, the data is going to be stored in Svelte stores.
The complete code can be found in the cloud37 github repository, the following code is just a partial example to reduce complexity.
// Function to check if a zipObject is a texture
const isTexture = (zipObject: JSZipObject) =>
   // Directories can't be textures
   !zipObject.dir &&
   // Item textures
   (zipObject.name.includes('src/main/resources/assets/ars_nouveau/textures/items') ||
      // Images in Patchouli Book
      zipObject.name.includes('src/main/resources/assets/ars_nouveau/textures/gui/entries'));

export const getTextureFiles = (zip: JSZip) => {
   return Promise.all(
      // Filter texture files out of all the files contained in the zip
      Object.values(zip.files)
         .filter(isTexture)
         // Map the zip objects to their base64 encoded file content 
         .map((file) =>
            zip.file(file.name)?.async('base64')
               .then((value) => {
                  // Write the encoded content into an object with the filename as key
                  return { [getFileName(file.name)]: value };
               })
         )
   ).then((resultingArray) => {
      // Reduce the Array of { filename: filecontents } objects to a dictionary
      return resultingArray.reduce((previousValue, currentValue) => {
         return { ...previousValue, ...currentValue };
      }, {});
   });
};

// in +layout.ts
.then((blob) => {
   return JSZip.loadAsync(blob);
})
// Pass the initalized Zip object to getTextureFiles
.then(getTextureFiles)
.then(loadedTextures => {
      // Set the texture store to the dictionary returned from getTextureFiles
      texturesStore.set(loadedTextures);
      // Indicate that stores have been initialized and don't need to be loaded again
      storesLoaded.set(true);
      setTimeout(() => {
         storesLoaded.set(false);
      }, 86400000 /* stores should be refreshed after 1 day if the server is on for so long */);
   }
);


		

In the next post, we’ll take a look at how to build a beautiful web page with the contents we’ve just loaded and prepared from the zipped repository and reveal the final result!