Natural Farming and the Data Vault

For some reason, I have a very keen interest in farming and farm practices (even though, I’ve never really owned a farm or even worked on one). My country of birth is essentially an agri-economy and the world’s second largest producer of agriculture, so maybe it’s tied to my roots.

Nevertheless, I was just browsing a farming forum and there appeared to be intense debates between the proponents of agriculture using chemicals and the organic farmers.

The debate was fairly intense with both sides citing scientific evidence, studies, pictures, proofs, and even name calling.

Kind of reminded me of the Inmon/Kimball divide in Data Warehousing.

In essence, it came down on one side: to the “promise” of quicker gratification – in the case of using chemical fertilizer and pesticides which takes a toll on the soil quality every 2-3 years. Work is required to fix the soil quality and essentially start from scratch pretty much every few years. This felt rather similar to common issues with the Kimball based architecture, which becomes unsustainable after a few iterations due to the expense of incorporating changes.

Just like the completely nutrition depleted soils require major fixing.

On the other hand, there’s organic farming which is a much more long-term solution, since it doesn’t deteriorate the soil quality, but it comes at a considerable expense, mostly of work.

Now, this is rather curious (and a rather bizarre coincidence too) …

My neighbours brother-in-law is an award winning organic farmer who doesn’t use chemical fertilizers and pesticides and he was in town. I happened to chat with him and he said, it’s a much more sustainable practice no doubt, but to remain profitable, he needs to sell his organically grown produce at approximately double the cost of the other.

This reminded me a bit of the old school Inmon architecture where upfront work is required, despite it’s longevity. It’s also associated with higher build costs (mainly because people don’t figure out long-term costs). While we can debate this, lets go with the perception for now.

It’s funny how we all end up relating almost everything to our own field of work and expertise.

And, then there is this thing called “Natural Farming”:

As the discussions continued, I was rather intrigued by something called “natural farming”, where they take crops and create an environment to mimic growth of vegetation in forests with minimal input. Surely, some prep-work is required and it’s based on patterns and combinations, just like the Data Vault, but once it’s all set up, there’s really no need to touch anything.

It was interesting and led me to another rather interesting pattern where they use crop combinations in small 9×9 size plots which fit into 36×36 plots, and all the way up to an acre. It looks different and it’s hard to get our head around it, despite it’s precision and patterns since it mimics nature.

But, the order and scalability also reminded me of the Data Vault.

I saw an interesting case study and implementation of it, with various rules of what combinations of crops to use and the profitability they would bring. And, the patterns were rather interesting considering as there were various potential combinations which would depend on your local native plants and their varieties.

Obviously, there’s quite a bit of opposition on the “natural farming” by “experts”, but the actual farmers are very happy because it’s profitable for them and requires very little work after initial setup.

It can also be done in an “agile” manner, where they pick a small parcel of land and go from nothing to forest first, and only then replicate it the the other parts of their land.

“Natural Farming” reminded me so much of the Data Vault Architecture, because:

– It’s a long-term solution

– It can be implemented quickly as a test on a small component end-to-end (Agile as recommended)

– It can be built out as blocks

– Requires very little maintenance after initial set up

– Provides value from the very first implementation even if it’s very small

– Can be tested small before committing and nothing built needs to be touched when scaling out (at least in the DV based DW)

– It’s not mainstream, despite the successes

– Tries to mimic elements from nature

– Requires people to have the right training (from the right people) so they do it right the first time

In fact, the naysayers were the people of other methods, who haven’t even tried it … or the people who have a vested interest in it continuing. This includes companies who produce chemical fertilizers and pesticides, and many “experts” from schools where studies are “sponsored” either directly or through endowments, and in many cases … numbers manipulated (As a BI geek, it’s soooo easy to see through these).

Many of the largest consulting companies don’t like the Data Vault, because you can build it fast, and leave the client. It’s not the cash-cow they’ve associated most Business Intelligence projects with, where they just continue to churn work, and continue milking the client with a giant team of their contractors on or off site [More on this soon]

Some have even tried to defame it.

On the other hand, the best Data Vault consultants work in the best interest of their clients, providing solutions, helping them build out fast and moving on. This is not to say, a DV project can’t be a large effort spanning time, but the actual usable output by time periods is vastly superior to anything else out there because of its nature and longevity.

It’s one of the reasons it’s been working for so long (Dan invented it in 1990 and released it to the world in 2001).

Data Vault 2.0 was a natural evolution (and completely dwarfs 1.0), but we’ll get to that in a different article.

And, Another interesting observation …

The very small percentage of failures of these natural farmers ended up being distributed between … predictions by people who simply had not implemented anything, but had their own “expert opinions”, and they were called out by some of the experts as “arm chair farmers” or people who just were not trained correctly.

I was an observer, on the forum I was reading, but you could see the real experts letting them know exactly what the issues were, with detailed explanations as they had considerable experience.

And, talking about getting connected to the right people, you have to check out the WWDVC where you’ll not only interact directly with the inventor of Data Vault and DV 2.0, but a whole bunch of other people in the field doing this day in and day out.

Imagine the stories you could swap.

The sessions are absolutely brilliant.

[ WWDVC Schedule ]

Getting back to the topic of farming, it was interesting to observe parallel patterns to say the least. It was also curious to hear an oxymoronic term like natural farming, but still watching the superiority and longevity of any solution that borrows from or mimics elements of nature.

The inspiration for the Data Vault components came from storage in the human brain as you may already be aware of.

Until next time