Use WordPress for data processing #1

Environment

The amount of available time was very limited and there was no budget to pay for external tools. But a host with pre installed and running Docker was at hand, which offered some free resources.

Getting the data

After running wordpress inside a docker container on that host, I installed the processing plugins. There were no surprises and everything went smooth.

After reading and understanding the documentation of the plugins, a bit trial and error I figured out the correct configuration.

The correct results were visible inside WordPress *yay*. Little did I know that this was to shortest and easiest part …

Prototype 1

After the data processing setup I did the first try to extract the data. Searching on Google I found WPGraphQL which allows you to use regular GraphQL Queries to retrieve data from WordPress.

Could it be so easy? Install WordPress, add the plugin for GraphQL and query the data from the main project?

Turns out, it´s not that simple. The plugin for data processing creates custom post types, which are not available via the GraphQL Query.

Researching how to add custom queries I found that you need to create a plugin, which will only be an extension for WPGraphQL.

After finding that out I copied and pasted together an extension, which does the trick. Finally I can query my data *yay*.

Added the new query to the main system and boom, it´s loading.

Sadly querying it shows that it is only 3/4 of the story. I receive the new field, but the wrong post type.

So I learned how to expose a custom post type too. After I finally got it right, I´m querying the new Post type.

But 2 minutes later I get an error, a timeout :(.

As timeouts are a no-go this isn´t a feasible option anymore. It might be that the internet connection to my local machine is temporarily too bad and it would normally pan out as it would be a connection between two data centers.

But I can´t rely on that. Also it feels weird that I have to do a pull approach, using up resources if there was a change or not.

Therefore I decided to change my approach and research how a push can be done from within WordPress.

Stay tuned for part 2.

Let me know how you likes article.

Best,

Frank

Links

https://wordpress.org/plugins/

https://www.wpgraphql.com/

Leave a Reply