This will be a different kind of post. One that hopefully shows you an alternative and inspires new ideas or approaches.
Problem
For a side project that I wanted to do as an experiment I needed data from multiple different websites. In the first version ~40 Websites were to be used.
Environment
The amount of available time was very limited and there was no budget to pay for external tools. But a host with pre installed and running Docker was at hand, which offered some free resources.
Analysis
As I was familiar with wordpress and had seen some of the plugins that can almost completely turn around it´s functionality, I googled for potential candidates.
And sure enough there were plugins available which allowed me to do the work for me.
But then there is the problem how to get the data out of wordpress, once it is in there.
So the search continued. How can one extract data from ones own wordpress installation?
Finally I stumbled over WPGraphQL, which allows you to use regular GraphQL Queries to retrieve data from WordPress.
Prototype 1st Try
Could it be so easy? Install WordPress, add the plugins for the data processing, add the plugin for GraphQL and query the data from the main project?
Turns out, it´s not that simple. The plugin for data processing creates custom post types, which are not available via the GraphQL Query.
Read more about it here
Prototype 2st Try
Researching how to add custom queries I found that you need to create a plugin, which will only be an extension for WPGraphQL.
After finding that out I copied and pasted together an extension, which does the trick. Finally I can query my data *yay*.
Added the new query to the main system and boom, it´s loading.
But 2 minutes later I get an error, a timeout :(.
Read more about that here.
Prototype 3rd Try
To be continued (So far the project stays at the 2nd try).
Let me know when this article helped you.
Best,
Frank