Blogs

How To Use a Static-site Generator: Build a Website With Gatsby and dotCMS

Freddy Montes

If you’re looking to launch a small, static, and speedy website, micro-site, or landing page; you may be considering a static-site generator like Gatsby.

This article will walk you through the process of using Gatsby alongside dotCMS, a Java-based open-source headless CMS, to build a static website that uses API calls to pull content that’s created, stored, and managed in dotCMS, and presented by Gatsby.

What You Need:

NodeJS and NPM
ReactJS knowledge
GraphQL knowledge (not required but you are going to use it. You can copy + paste)

Gatsby define themselves as “a blazing fast modern site generator for React” and yes, the generated sites are fast out of the box.

Gatsby’s rich data plugin ecosystem lets you build sites with the data you want — from one or many sources: Pull data from headless CMSs (like dotCMS), SaaS services, APIs (like the dotCMS content API) and the allows you to bring the data into your page using GraphQL.

Static-site Generator Benefits

Security

No databases and the code injection threat is close to none.

Reliability

You can serve html files everywhere.

Speed

Regular static sites are fast (again no database or backend) and with Gatsby + React they are even faster.

Scalability

No need complex on the server, static websites with HTML files can be easily scaled up by just increasing the bandwidth

Static-site Generator Limitations

Fetching data for a Gatsby site happens at build time, meaning that if new content is created in dotCMS, it will be not available into your site until the next build, or the next time you “run” your static-site generator CMS—which is Gatsby in today’s case.

But… there are solutions for this too:

You can create a cronjob that run every N minutes to build and deploy your site.
Hybrid app pages to the rescue. Remember Gatsby have ReactJS in his core which means you can make network request to APIs and get new data into your components.

How to Create a Gatsby site

Install Gatsby

Gatsby have this amazing cli tool to create, manage and run Gatsby projects, to install it, go to your terminal and run:

$ npm install --global gatsby-cli

If everything went well, you can type:

$ gatsby -v

And get 2.4.3 or a similar version:

gatsby-1

Create a Gatsby site

Now let’s use Gatsby cli tool to create a site:

$ gatsby new dotcms-site

Note: You can replace “dotcms-site” with the name you want for your project.

The command you just ran did:

Create a new site with the Gatsby default starter
Create a folder with the name of the project (dotcms-site)
Install all the npm packages that needs to run the site
Make your life easier :)

Let’s run our new Gatsby site:

$ cd dotcms-site

Browse into the folder of your project.

$ gatsby develop

This command starts a development server. You will be able to see and interact with your new site in a development environment. Also it have live reload, so any changes you do in your files you can see immediately in your site.

Now open a browser and go to http://localhost:8000 if everything went well you should something like:

gatsby-starter-site-hi-people

To Get Data, You Need a Source Plugin

Gatsby have a plugin system, in order to get data you need what they call a: “Source Plugin”. Source plugins “source” data from remote or local locations into what Gatsby calls nodes.

Think of a node to be the exact equivalent to a contentlet in dotCMS. You’re going to write a Gatsby source plugin that fetch all the contentlets in a dotCMS instance and turn them into Gatsby nodes that you can later display in our pages by querying with GraphQL.

gatsby-blog-graphic

Before start, if you like to read more about Gatsby Source plugin their documentation and tutorial is really good.

Create gatsby-source-dotcms plugin

The bare essentials of a plugin are: directory named after your plugin, which contains a package.json file and a gatsby-node.js file:

|-- plugins
    |-- gatsby-source-dotcms
        |-- gatsby-node.js
        |-- package.json

Start by creating the directory and changing into it:

$ mkdir plugins
$ mkdir plugins/gatsby-source-dotcms
$ cd plugins/gatsby-source-dotcms

Create a package.json file

Now create a package.json file, this describes your plugin and any third-party code it might depend on. npm has a command to create this file for you. Run:

$ npm init --yes

to create the file using default options.

NOTE: You can omit --yes if you’d like to specify the options yourself.

With the setup done, move on to adding the plugin’s functionality.

Create a gatsby-node.js file

Create a new file called gatsby-node.js in your gatsby-source-dotcms directory. Open the file in your favorite code editor and add the following:

exports.sourceNodes = ({ actions, createNodeId, createContentDigest }, configOptions) => {
  const { createNode } = actions

  // Gatsby adds a configOption that's not needed for this plugin, delete it
  delete configOptions.plugins

  // plugin code goes here...
  console.log("Testing my DotCMS plugin", configOptions)
}

What did you do by adding this code?

You implemented Gatsby’s sourceNodes API which Gatsby will run as part of its bootstrap process. When Gatsby calls sourceNodes, it’ll pass in some helper functions (actions, createNodeId and createContentDigest) along with any config options that are provided in your project’s gatsby-config.js file:

exports.sourceNodes = ({ actions, createNodeId, createContentDigest }, configOptions) => {...}

You do some initial setup:

const { createNode } = actions

// Gatsby adds a configOption that's not needed for this plugin, delete it
delete configOptions.plugins

And finally add a placeholder message:

console.log("Testing my DotCMS plugin", configOptions)

How to Add the dotCMS Plugin to Your Gatsby Site

The skeleton of your plugin is in place which means you can add it to your project and check your progress so far.

Open gatsby-config.js from the root directory of your tutorial site, and add the gatsby-source-dotcms plugin:

module.exports = {
  siteMetadata: {
    title: "Gatsby Default Starter",
  },
  plugins: [
    {
      resolve: "gatsby-source-dotcms",
      options: {},
    },
  ],
}

Open a new terminal in the root directory of your tutorial site, then start Gatsby’s development mode:

$ gatsby develop

Check the lines after success on PreBootstrap, you should see your “Testing my plugin” message along with an empty object from the options your gatsby-config.js file:

testing-my-plugin

Note that Gatsby is warning that your plugin is not generating any Gatsby nodes. Time to fix that.

Getting the data from DotCMS

Like I mention before, you need to get ALL the contentlets from the dotCMS instance and turn them into Gatsby nodes.

The action plan

In order to create a GraphQL node that you can query by Content Type, you need two things from dotCMS:

The contentlet
The content type of each contentlet

In DotCMS you need to use the new Content Types rest endpoint to get all the content types variables and then use the Content API endpoint to get all the contentlet for each content type in the instance.

You’ll combine the results from the two requests together to create a big collection of contentlets with an extra property of contentType.

Add dependencies

From your plugin folder: /plugins/gatsby-source-dotcms run:

$ npm install node-fetch --save

Open your package.json file and you’ll see node-fetch have been added to a dependencies section at the end:

"dependencies": {
    "node-fetch": "^2.2.0"
}

node-fetch is a light-weight module that brings window.fetch to Node.js so you can use fetch to do the request to DotCMS endpoints.

Create a dotCMS javascript library

To create the DotCMS library to get all the contentlets and content types you going to use:

I’m assuming that you are familiar with this technologies, so I’m not going to explain too deep how they work in this tutorial.

To begin, create a dotcms-api.js file inside the /plugins/gatsby-source-dotcms open the file and add the following code:

const fetch = require('node-fetch')

class DotCMSApi {
    constructor(options) {
        this.options = options
    }

    getBaseUrl() {
        return `${this.options.host.protocol}://${this.options.host.url}`
    }

    getContentletsByContentType(contentType) {
        const getUrl = () => {
            return `${this.getBaseUrl()}/api/content/render/false/query/+contentType:${contentType}%20+(conhost:${
                this.options.host.identifier
            }%20conhost:SYSTEM_HOST)%20+languageId:1%20+deleted:false%20+working:true/orderby/modDate%20desc`
        }

        return fetch(getUrl())
            .then(data => data.json())
            .then(data => data.contentlets)
            .then(contentlets => {
                contentlets.forEach(contentlet => {
                    contentlet.contentType = contentType
                })
                return contentlets
            })
    }

    async getContentTypesVariables() {
        const getUrl = () => {
            return `${this.getBaseUrl()}/api/v1/contenttype?per_page=100`
        }

        return fetch(getUrl(), {
            headers: {
                DOTAUTH: Buffer.from(
                    `${this.options.credentials.email}:${this.options.credentials.password}`
                ).toString('base64'),
            },
        })
            .then(data => data.json())
            .then(contentTypes => contentTypes.entity.map(contentType => contentType.variable))
    }

    async getData() {
        const contentlets = await this.getContentTypesVariables().then(variables => {
            return variables.map(async variable => {
                const data = await this.getContentletsByContentType(variable)
                return data
            })
        })

        return Promise.all(contentlets)
    }
}

exports.getContentlets = async configOptions => {
    const dotCMSApi = new DotCMSApi(configOptions)

    return dotCMSApi.getData().then(contentTypesContentlets => {
        // Flatten nested array
        return [].concat.apply([], contentTypesContentlets)
    })
}

What is all this code doing?

const fetch = require('node-fetch')

You imported the npm module node-fetch it will be use to do the requests to the dotCMS instance and get the data you need.

class DotCMSLibrary {
    constructor(options) {
        this.options = options
    }

Then you created a class that contains the methods you use to get the data you need (contentlets and content types). When you create an instance of this class you pass the options that you’ll add to the gatsby-config.js file.

getBaseUrl() {
    return `${this.options.host.protocol}://${this.options.host.url}`
}

Simple method to get the dotCMS instance url.

getContentletsByContentType(contentType) {
    const getUrl = () => {
        return `${this.getBaseUrl()}/api/content/render/false/query/+contentType:${contentType}%20+(conhost:${
            this.options.host.identifier
        }%20conhost:SYSTEM_HOST)%20+languageId:1%20+deleted:false%20+working:true/orderby/modDate%20desc`
    }

    return fetch(getUrl())
        .then(data => data.json())
        .then(data => data.contentlets)
        .then(contentlets => {
            contentlets.forEach(contentlet => {
                contentlet.contentType = contentType
            })
            return contentlets
        })
}

In this method you do the request to get all the contentlets of the specific content type is pass as parameter, also to each contentlet you add a property of contentType that will be used to query data. This method is called several times, once for each content type in the dotCMS instance.

async getContentTypesVariables() {
    const getUrl = () => {
        return `${this.getBaseUrl()}/api/v1/contenttype?per_page=100`
    }

    return fetch(getUrl(), {
        headers: {
            DOTAUTH: Buffer.from(
                `${this.options.credentials.email}:${this.options.credentials.password}`
            ).toString('base64'),
        },
    })
        .then(data => data.json())
        .then(contentTypes => contentTypes.entity.map(contentType => contentType.variable))
}

This method will get an array of content types variables using the new Content Types API and then map the result to an array of just variables.

NOTE: make sure you pass the query param per_page=100 at the url with the full amount (or more) of content types in your instance.

async getData() {
    const contentlets = await this.getContentTypesVariables().then(variables => {
        return variables.map(async variable => {
            const data = await this.getContentletsByContentType(variable)
            return data
        })
    })

    return Promise.all(contentlets)
}

And finally you used the getContentTypesVariables and getContentletsByContentType to get all the contentlets in the dotCMS instance, first you get an array of content types variables and for each one of them you do a request to the dotCMS Content API to get the all contentlets for each content type. Remember that this code runs on build time which mean it not going to be a performance issue doing so many requests.

The constant contentlets results in an array of Promises of contentlets so you have to use Promise.all to returns a single Promise that resolves when all of the contentlet promises have resolved.

exports.getContentlets = async configOptions => {
    const dotCMSApi = new DotCMSApi(configOptions)

    return dotCMSApi.getData().then(contentTypesContentlets => {
        // Flatten nested array
        return [].concat.apply([], contentTypesContentlets)
    })
}

Finally you export one method from this library, here you initialize the DotCMSApi with the config options from the gatsby-config.js file get all the contentlets in the DotCMS instance and flat it into one big collection of contentlets with an extra property of content type, the result will be like:

[
    {
        "contentType": "newsItem",
        "owner": "dotcms.org.1",
        "identifier": "f60ed48b-1f5f-4a7b-b4b0-f5a857b41e6a",
        "inode": "734944ff-6f02-4337-b9fe-aef3c372dad8",
        "title": "This is a new item",
        "expire": "2020-01-02 02:19:00.0",
        "tags": "oil,investment,gas,prices,retiree:persona"
    },
    {
       ...
    },
   ...
]

The appended contentType property in this collection of contentlets is key, because you’ll be using that to do our GraphQL queries.

Writing the Gatsby Plugin

Open the /gatsby-source-dotcms/gatsby-node.js file and replace the code with the following:

const dotCMSApi = require('./dotcms-api');

exports.sourceNodes = ({ actions, createNodeId }, configOptions) => {
    const { createNode } = actions

    // Gatsby adds a configOption that's not needed for this plugin, delete it
    delete configOptions.plugins

    // Helper function that processes a contentlet to match Gatsby's node structure
    const processContentlet = contentlet => {
        const nodeId = createNodeId(`dotcms-${contentlet.contentType}-${contentlet.inode}`)
        const nodeContent = JSON.stringify(contentlet)

        const nodeData = {
            ...contentlet,
            id: nodeId,
            parent: null,
            children: [],
            internal: {
                type: `DotCMS${contentlet.contentType}`,
                content: nodeContent,
                contentDigest: JSON.stringify(contentlet),
            },
        }

        return nodeData
    }

    // Gatsby expects sourceNodes to return a promise
    return (
        dotCMSApi.getContentlets(configOptions)
            .then(contentlets => {
                // Process the response data into a node
                contentlets.forEach((contentlet) => {
                    // Process each contentlet data to match the structure of a Gatsby node
                    const nodeData = processContentlet(contentlet)

                    // Use Gatsby's createNode helper to create a node from the node data
                    createNode(nodeData)
                })
            })
    )
}

Let’s go over the new code

const dotCMSApi = require('./dotcms-api');

First at all, you imported the dotCMS API you created.

const processContentlet = contentlet => {
    const nodeId = createNodeId(`dotcms-${contentlet.contentType}-${contentlet.inode}`)
    const nodeContent = JSON.stringify(contentlet)

    const nodeData = {
        ...contentlet,
        id: nodeId,
        parent: null,
        children: [],
        internal: {
            type: `DotCMS${contentlet.contentType}`,
            content: nodeContent,
            contentDigest: JSON.stringify(contentlet),
        },
    }

    return nodeData
}

The job of this function is to receive a dotCMS contentlet and return a Gatsby Node by extending it with a spread operator. The basic node structure looks like:

id: String,
children: Array[String],
parent: String,
// Reserved for plugins who wish to extend other nodes.
fields: Object,
internal: {
    contentDigest: String,
    // Optional media type (https://en.wikipedia.org/wiki/Media_type) to indicate
    // to transformer plugins this node has data they can further process.
    mediaType: String,
    // A globally unique node type chosen by the plugin owner.
    type: String,
    // The plugin which created this node.
    owner: String,
    // Stores which plugins created which fields.
    fieldOwners: Object,
    // Optional field exposing the raw content for this node
    // that transformer plugins can take and further process.
    content: String,
}
...other fields specific to this type of node

// Gatsby expects sourceNodes to return a promise
return (
    dotCMSApi.getContentlets(configOptions)
        .then(contentlets => {
            // Process the response data into a node
            contentlets.forEach((contentlet) => {
                // Process each contentlet data to match the structure of a Gatsby node
                const nodeData = processContentlet(contentlet)

                // Use Gatsby's createNode helper to create a node from the node data
                createNode(nodeData)
            })
        })
)

The final piece of the puzzle is to return a Promise of processed Gatsby nodes. You used the dotCMSApi library to getContentlets and convert each one of those contentlets you got into a node.

Trying our fancy new plugin

Edit your gatsby-config.js and update the entry for gatsby-source-dotcms:

{
    resolve: 'gatsby-source-dotcms',
    options: {
        host: {
            protocol: 'http',
            url: 'localhost:8080',
            identifier: '48190c8c-42c4-46af-8d1a-0cd5db894797'
        },
        credentials: {
            email: 'admin@dotcms.com',
            password: 'admin',
        },
    },
}

NOTE: This configuration is set for a dotCMS instance running in localhost:8080, make sure you point this values to an actual running dotCMS instance.

Now from the root of your project, run in your Terminal:

$ gatsby develop

If everything went well you should see:

gatsby-done

Querying DotCMS data with GraphQL

Every time you run $ gatsby develop you get this:

If you go to that url: http://localhost:8000/___graphql in your browser you will get a webapp that allows you to run GraphQL queries and make sure all DotCMS content is there, it will look like this:

graphiql

How to Generate Pages

Note: As I mention before Gatsby use ReactJS to build pages and component, is this tutorial I am assuming you are familiar with this library.

So you have ALL our dotCMS content ready for use. Let’s create some pages.

You can create pages in Gatsby explicitly by defining React components in src/pages/, or programmatically by using the createPages API.

Create a news listing page

By default Gatsby creates a src/pages/page-2.js let’s rename that file to src/pages/news.js and then replace the code with:

import React from 'react'
import { Link } from 'gatsby'

import Layout from '../components/layout'

const NewsPage = () => (
  <Layout>
    <h1>News list</h1>
    <p>Here we'll show a news list</p>
    <Link to="/">Go back to the homepage</Link>
  </Layout>
)

export default NewsPage

This is very simple ReactJS code, you just created a component that will print out plain HTML.

Let’s see this page, go to your Terminal and run:

$ gatsby develop

And then open your browser to: http://localhost:8000/news and you should see:

gatsby-default-starter

Bring the dotCMS contentlets

Okey, now is a good time to use the source plugin you built before, you’re going to query all the contentlets of the content type “News” and put them in our page.

Once again edit src/pages/news.js and add the following code:

import React from 'react'
import { graphql } from 'gatsby'
import Layout from '../components/layout'

const NewsPage = ({ data }) => (
  <Layout>
    <h1>News list</h1>
    <ul class="news">
        {data.allDotCmsNews.edges.map(({ node }, index) => (
            <li key={index}>
                <h4>{node.title}</h4>
                <p>{node.lead}</p>
            </li>
        ))}
    </ul>
  </Layout>
)

export const query = graphql`
    query {
        allDotCmsNews {
            edges {
                node {
                    lead
                    title
                    urlTitle
                }
            }
        }
    }
`

export default NewsPage

Let’s go over this new code

import { graphql } from 'gatsby'

You need to import graphql so you can use to query our contentlets. Gatsby’s graphql tag enables page components to retrieve data via GraphQL query.

const NewsPage = ({ data }) => (

The data prop contains the results of the GraphQL query, and matches the shape you would expect.

<ul class="news">
    {data.allDotCmsNews.edges.map(({ node }, index) => (
        <li key={index}>
            <h4>{node.title}</h4>
            <p>{node.lead}</p>
        </li>
    ))}
</ul>

Finally you just print out the data you get from the GraphQL query.

export const query = graphql`
    query {
        allDotCmsNews {
            edges {
                node {
                    lead
                    title
                    urlTitle
                }
            }
        }
    }
`

Here is where the “magic” of getting the data happen, is a simple GraphQL query to get all the dotCMS News items, in this case you asked for the field: lead, title and urlTitle. If everything went good, you should see something like:

gatsby-default-starter-2

Creating news detail pages

You have a list of news, now let’s create a detail page for each news item in DotCMS. I know that’s sound like a ton of works, but with Gatsby you can programmatically create pages from data: basically you can tell Gatsby, here is a collection of news and create a page for each item using this template (another react component).

Edit your gatsby-node.js file and add the following code:

const path = require(`path`)

exports.createPages = ({ graphql, actions }) => {
    const { createPage } = actions
    return new Promise((resolve, reject) => {
        graphql(`
            {
                allDotCmsNews {
                    edges {
                        node {
                            inode
                            lead
                            sysPublishDate
                            title
                            urlTitle
                        }
                    }
                }
            }
        `).then(result => {
            result.data.allDotCmsNews.edges.forEach(({ node }) => {
                createPage({
                    path: `news/${node.urlTitle}`,
                    component: path.resolve(`./src/templates/news-item.js`),
                    context: {
                        // Data passed to context is available
                        // in page queries as GraphQL variables.
                        slug: node.urlTitle,
                    },
                })
            })
            resolve()
        })
    })
}

What is this code doing?

First at all, this code will be running at build time, all this pages will be generated once when you build Gatsby, so if new contentlet are added to DotCMS you need to build and deploy your Gatsby site in order to get that content.

const path = require(`path`)

This is the implementation of the createPages API which Gatsby calls so plugins can add pages. By exporting createPages from gatsby-node.js file, you’re saying, “at this point in the bootstrapping sequence, run this code”.

graphql(`
    {
        allDotCmsNews {
            edges {
                node {
                    inode
                    lead
                    sysPublishDate
                    title
                    urlTitle
                }
            }
        }
    }
`

The same as you did with the listing page, here is another GraphQL query to get all the DotCMS news contentlets, the only difference is that you’re asking for more fields.

.then(result => {
    result.data.allDotCmsNews.edges.forEach(({ node }) => {
        createPage({
            path: `news/${node.urlTitle}`,
            component: path.resolve(`./src/templates/news-item.js`),
            context: {
                // Data passed to context is available
                // in page queries as GraphQL variables.
                slug: node.urlTitle,
            },
        })
    })
    resolve()
})

Once the GraphQL query resolver you have access to all the news item, youiterate over and create a page for each item, youneed to pass an object to the createPage function:

path: the url where the generated page will be create, in this case it will be news/page-url-title, youuse the urlTitle field from the contrentlet to create the path, whatever you use, needs to be unique, because this will be the name of the html generated file and by means the url of the page.
component: this is where youtell Gatsby which template will be using to create each page (youwill create this in the next step)
context: this is an object that youare passing to the GraphQL query will be doing in the template component.

Finally youcall resolve() for the Promise where returning in the implementation of the createPages function.

Creating the template

A template is just a regular ReactJS component, so let’s create a file in: src/templates/news-item.js (like youset in the path property of the createPage param) and add the following code:

import React from 'react'
import { graphql } from 'gatsby'
import Layout from '../components/layout'

export default ({ data }) => {
    const post = data.allDotCmsNews.edges[0].node
    return (
        <Layout>
            <h1>{post.title}</h1>
            <div dangerouslySetInnerHTML={{ __html: post.story }} />
        </Layout>
    )
}

export const query = graphql`
    query($slug: String!) {
        allDotCmsNews(filter: { urlTitle: { eq: $slug } }) {
            edges {
                node {
                    title,
                    story,
                }
            }
        }
    }
`

export default ({ data }) => {

As you get it before the data to use in the page you get it as a prop in the component.

<Layout>
    <h1>{post.title}</h1>
    <div dangerouslySetInnerHTML={{ __html: post.story }} />
</Layout>

You print out the content of the news item in the page.

export const query = graphql`
    query($slug: String!) {
        allDotCmsNews(filter: { urlTitle: { eq: $slug } }) {
            edges {
                node {
                    title,
                    story,
                }
            }
        }
    }
`

Here is in the GraphQL query everything is almost the same as the others queries with the difference that you’re matching ONE news item by urlTitle and the $slug param is what you pass in the context object in the params of the createPage function in the gatsby-node.js

Now you can build our pages, just need to run:

$ gatsby develop

Right now you don’t have links to our recently created links, but you use the urlTitle field from the contentlet, so you can go to: http://localhost:8080/news/the-gas-price-rollercoaster and you should see:

Gatsby Rollercoaster

Final Touches: Adding Links to Our Listing Page

Go and edit src/pages/news.js and:

Import the Link component on the top of the file:

import { Link } from 'gatsby'

Replace this line:

<h4>{node.title}</h4>

With:

<h4><Link to={'news/' + node.urlTitle}>{node.title}</Link></h4>

In the browser go to http://localhost:8080/news and you should see:

gatsby-default-starter-news

Now the news list item are linked to the detail page you dynamically created with Gatsby.

Static-site Generator + Headless CMS = The Perfect Match

And there you have it! You’ve now used a static-site generator and a headless CMS to build a static website. This is just one example of how these two technologies can combine.

To recap, we built a Gatsby Source Plugin from scratch, created a listing page, and created a bunch of static html pages querying dotCMS data with GraphQL.

I recommend you take a look Gatsby Documentation, this is the very basic but you can do so much more, styled components, layouts, pagination, etc.