Family eating dinner

Are you making authentic supply chain promises?

If you’re in the food business (whether that’s retail, food service, processing, farming, or supply), consumers are asking questions about your supply chain.

Of course, they may not be asking you directly, and they may not be asking your retail or food service partner, but they are asking: on social media, on recommendation sites such as TripAdvisor and Yelp, even over drinks at their local.

Are you providing the information they need to be confident about the quality and safety of your product? Do you have a substantiated story around provenance, animal welfare and the environment?

Safeguards such as DNA testing lasagna are “bottom of the cliff” activities, an attempt to rebuild broken trust and arguably too limited and late in the supply chain.

Future product preference and even acceptance relies upon a supply chain that can show ethical practices: in how environmental impacts are managed, natural biodiversity is encouraged, animal welfare is maintained, anti-microbial resistance is avoided, and workers and communities are treated.

Activist groups and the power of social media means that our response to these demands must be much more solid than a promise or a declaration form. We must have the systems and measures to back up our words – and to demonstrate as much to auditors and our supply-chain partners.

For those of us at the confluence of technology and agriculture, this means we must do more than just record activities and calculate gross margins. We must step up with tools that capture rich data in support of farming activities, and which actively encourage good decisions that improve both profitability and sustainability.

All this needs to be done with minimal additional effort by farmers and their staff, and aligned to real-world processes on farm.

I’ll be speaking at MobileTech 2017, the annual summit for technology innovations in the primary sector, reflecting on these challenges. I’ll summarise some of the work Rezare Systems is doing in this space, and suggest ways the industry could apply technology to the opportunity.

This article was first published at  

Dairy farmer with technology

Farmers love technology, fear misuse

Increasing numbers of farmers see technology as useful and important to their farming businesses, and farmers are looking to invest further in new technology over the coming years. Despite this, lingering concerns about data sharing, privacy and control remain.

According to the October 2016 Commonwealth Bank of Australia Agri-Insights Survey of 1600 Australian farmers, 70% of farmers believe that the digital technology available adds significant value to their businesses.

The Ag Data Survey published by the American Farm Bureau Federation (AFBF) also found that farmers are optimistic about technology, with 77% of farmers planning to invest in new technology for their farms in the next three years.

Farmers also see value in sharing and re-use of data, but privacy and control are the largest barriers to more widespread re-use.

The Agri-Insights Survey found that:

  • 76% of farmers think that there is value in sharing on-farm production information with others;
  • 58% of farmers currently share some on-farm production information with others; and
  • Of farmers who don’t see value in data sharing, “privacy concerns” at 28% is the largest reason.

The New Zealand Office of the Privacy Commissioner surveyed New Zealanders about privacy and their attitudes to data sharing in April 2016. They noted that:

  • 57% of respondents were open to sharing data if they could choose to opt out;
  • 59% were open to sharing if there were strict controls on who could access data and how it was used; and
  • 61% were open to sharing if the data was anonymised and they couldn’t be personally identified.

The US AFBF survey also highlighted some of these concerns in an agricultural context:

  • Only 33% of farmers had signed contracts with their ag-tech provider. Another 39% knew of their provider’s policies but had not signed anything;
  • When farmers were asked if they were aware of the ways in which an ag-tech provider might use their data, 78% of farmers answered “no”; and
  • 77% of farmers were concerned about which entities can access their farm data and whether it could be used for regulatory purposes.

Not just farmers

Confidentiality and control can be barriers to companies too. After all, much of the data is about their activities, products, or equipment as well as the farm itself.

It’s not always clear how other parties will behave when sharing data. Organisations generally make reasonable and effective use of data and meet confidentiality expectations, but there is always a risk that they won’t. So companies sharing data are forced to negotiate “iron-clad” agreements, keeping the corporate lawyers busy and making any new data exchange the subject of long-winded negotiations.

As soon as you get into negotiations like this, costs rise. If one of the parties is a smaller player with less negotiating power (company or farmer), they may never be able to conclude a useful data access deal. The end result? A slower rate of innovation, the benefits of information to the farmer and overall supply chain are not fully realised, and sharing data becomes a much more expensive exercise than you would otherwise expect.

Over the years, industry players have experimented with different ways to address these issues. Centralised industry-good databases and exchanges have been proposed, and these could be very effective. Unfortunately, concern about centralising large amounts of data, and the loss of control that this brings has led players to hold back some or all of their data from such repositories.

Other groups have posited that all data should be in the exclusive control of the farmer, and have built exchanges or created open API standards on that basis. We applaud this, but it doesn’t always reflect the significant effort that companies and service providers invest in creating and curating some data sets. The end result is that some data sets are often held back from such exchanges.

A collaborative approach

The New Zealand primary industry has worked on several approaches to this problem in a collaboration between the red meat sector, the dairy sector, and the Ministry for Primary industries.

The Farm Data Code of Practice is designed to encourage greater transparency between farmers and service providers or vendors about the data that is held, and the rights that each party has to the data. A straight-forward accreditation process gives farmers confidence that organisations have “got their house in order” when it comes to terms and conditions and data policies.

The DataLinker protocol builds on the standardised, open API approach to sharing data, but with three key considerations:

  • It provides a way for organisations to agree a Data Access Agreement without a protracted legal negotiation. Standard agreements are provided and encouraged, to reduce the overhead that all parties face in legal costs and time (that said, custom agreements are still possible where absolutely necessary).
  • Accepting a Data Access Agreement doesn’t give the recipient “open slather” to the data; for most data sets, explicit farmer approval is also required, requested and confirmed by the farmer using standard web authorisation protocols. Farmers grant permission to access data that covers their business, and can also withdraw that authorisation.
  • As an Open API approach is used rather than a central database or exchange, there is no “central service” that must be involved in each data transfer. This reduces the “attack surface” from a security perspective and enables organisations to retain control of the data they hold.

Organisations adopting the DataLinker protocols benefit in several ways:

  • Farmers see that they are playing their part in maximising the use of information;
  • Standardised APIs and Data Access Agreements reduce the time and money invested in negotiating and creating custom solutions for every interaction;
  • Data Access Agreements mean that companies still retain the necessary control over high-value data sets, and are able to meet the privacy and confidentiality terms they have agreed with farmers; and
  • Companies and farmers can efficiently use sets of data which otherwise might have been too expensive to collect, or required a level of farmer input which would have discouraged adoption.

Our hope is that this framework will help organisations and farmers to maximise use of farm information, reducing long-term costs and encouraging greater innovation.

What you can do about this:

  • Want to see which New Zealand companies are accredited under the Farm Data Code of Practice? Check out and drop an email to your key information providers to find out when they will be accredited.
  • Interested in the DataLinker protocols and how they can be adopted by your business? You’ll find information at
  • Planning your strategy in this data space, or considering next steps? Talk to us – we’re happy to provide you with background and advice.

How on-farm data and analysis can support credence attributes

Can on-farm technologies and “big data” support food and fibre product attributes that consumers value?

In a previous article I noted a Hartman Group study that suggested that consumers are interested in attributes other than just the look and price of a product, wanting to know:

  • What ingredients are in the food or beverage product (64%);
  • How a company treats animals used in its products (44%); and
  • From where a company sources its ingredients (43%).

We call these informational aspects of a product “credence attributes”, meaning that they give credence to our decision to purchase (or not purchase) a product or service, but can’t be directly assessed from the product itself, either before purchase (on the basis of colour or feel) or after purchase (on the basis of taste, for instance).

Characteristics such as “organic”, “environmentally responsible”, “grass-fed”, and “naturally raised” relate to the story behind a product. A product may communicate these through advertising, packaging, and other ways of telling the product story.

But consumers are also looking for authenticity and integrity in their food and other products. There’s a consumer backlash when the product story on the pack is in conflict with other data sources – such as claims in news articles or secret video footage.

We’ve been exploring ways that feeds of data from on-farm technology could be used to support the product provenance and credence story – or at least signal to farmers and their supply chain partners where checks and improvements should be considered. Here are a couple of examples.

Monitoring carbon footprint

Carbon life-cycle assessments (LCAs) are used to understand the extent to which production, manufacture, and distribution of a product impacts on climate change through deforestation or release of greenhouse gases such as carbon dioxide, methane, and nitrous oxide. We learn some interesting things from these, sometimes showing that shipping food products from the other side of the world can have a lower impact than growing products locally if the local environment is less hospitable.

Importantly, producing a Life-cycle assessment creates a model – a series of equations and if-then logic that describes the calculation. We can use this model with appropriate local farm and supply chain data to understand how management decisions and activities, timing and stock or crop productivity impact on emissions.

Automated systems on farms that capture data about crop production, livestock weights and production, and farm activities can also deliver data for a custom life-cycle assessment. Benchmark data across multiple farms and it becomes possible to identify the patterns of complete vs missing data, to understand how climatic constraints change emissions, or to identify outliers that need to be more closely examined.

A note of caution here: as we’ve learned from nutrient budgeting, farm systems can be varied and life-cycle assessment models are frequently based on the “typical”. An outlier result may indicate greater variation than the model can handle, rather than a more or less efficient farming system.

Demonstrating animal welfare

Animal welfare and the ability to live a healthy and natural life is another area of concern to consumers. Here too, metrics collected on-farm can be the subject of automated analysis to demonstrate good practices are followed.

In Europe where a premium is payable for “grass-fed” dairy in some regions, farmers are experimenting with the use of monitoring devices – smart tags and neck bands for example. These devices capture data that provide farmers with early warning of heats and potential animal health issues – raised temperatures, more or less movement, and reduced eating for example – but can also be analysed for patterns that only show up in outdoor grazing.

In other jurisdictions, veterinary product purchase, use, and reordering records can help to demonstrate compliance with animal health plans worked out between farmers and veterinarians, and hence demonstrate good welfare practices and appropriate use of medicines. Paper records have been used for this purpose for many years, but software technologies and automated data analysis can reduce the burden of data collection and the need for manual audits and analysis.

Practical application

Some producers will find the thought of such automated systems invasive and potentially threatening. Certainly, given the potential for outliers, for good practices that just don’t quite fit the expected mould, and for technology glitch or human error, you couldn’t use these measures as legal baselines that determine “rights to farm”.

Nevertheless, application of technology and analytics such as these can help us as we seek to improve farming practice and improve the integrity of our food supply chains. A good starting point might be to apply these as tools for committed producer groups that are already aligned with supply of a premium product or market.


This article was first published at
Contact us to learn how

we apply software and models to agricultural data.

Why invest in tech for farming?

Every second business writer today seems to be talking about ag-tech, big data, and the internet of things. If you’re an agriculture sector organisation, or a company servicing or purchasing from primary producers, you might be forgiven for thinking you missed a day in the office and overnight farming has become a connected, automated, artificially-intelligent system.

Reality of course is still far from that utopia (or dystopia, depending on your point of view). What the writers are telling us is that there are a range of interesting technical possibilities that might have application in agriculture: and that some of the very early adopters, enthusiasts, and visionaries are trialling these. In some cases, they’ve used technologies in interesting experiments and learned useful things about their supply chain or farming system.

All well and good; but if you don’t consider yourself a leading edge visionary (or perhaps you don’t have the same appetite for risk) should you just ignore the hype, and wait until the technology matures?

For the pragmatists among us, who are more interested in achieving practical goals and strategic goals than experimenting, here are some areas where I think there is value in leveraging technology into your business today.

Improving communication

We all know that good communication between people is what keeps the wheels of business oiled and turning. Technologies, systems, and business rules are no replacement for good people relationships.

Your customers, suppliers, and business partners see all of your interactions with them as part of that same interpersonal relationship. Are your reports or invoices late? Do they lack critical information your business partners need? Personal reassurances will go so far, but if you can’t achieve timely and data-rich information delivery that helps your partners, they may look elsewhere.

We’ve helped a number of our customers lift their communications with suppliers or customers. In some instances, we’ve delivered apps that help farmers access key pieces of information as they become available. In other cases, we’ve implemented reports and visualisations that are valued by business partners for the timely insight they provide. This isn’t just data: it’s business communication.

Understanding critical business metrics

Many businesses try and track too many metrics, and don’t always closely manage the key metrics that are leading indicators of success.

What drives your business? Primary production and processing businesses are often heavily influenced by factors such as weather and global market demand, but these factors are outside the control of most businesses and may have less impact on long-term profitability than we imagine.

Its typically our response to outside factors, and our ability to continue to produce value despite them that determines long term success. Measures of productivity per unit of input, and effectiveness at delivering high-value products are better indicators than raw dollar returns or kilograms of product shipped.

For some of our customers, this has meant improving alignment between their financial data and physical data records. For instance, benchmarking kilograms of product produced, farm working expenses and profit against the potential pasture or crop production for that season, allowing more effective comparison of improvements across seasons. For others it has meant focusing on the proportion of product meeting specification for high-value markets, regardless of whether the market actually delivered the desired price premium in that particular season.

Data integration can help bring these disparate data sets together for timely comparison, and in-field monitoring technologies or remote sensing can deliver the physical data needed to make sense of the product and financial outcomes.

Responding to changing conditions

How do you assess and respond to the risk of changing weather and markets?

Studies of farmer responses to drought and similar challenges indicate that we tend to respond too late, and in a conservative, “piecemeal” fashion. We seem to bet that things will trend back to normal sooner rather than later, and under-do our response.

Monitoring tools such as climate stations and market data visualisations allow us to understand trends and risks early – before their impacts really start to bite. Of course, responding to these is still a challenge: will I reduce stock numbers only to see the weather change?

Mathematical models don’t yet give those definitive answers some futurists might lead you to expect, but they allow you to ask the “what if” questions, looking at potential decisions and impacts. These allow you to build a plan for your business and understand what your critical review and decision points need to be.

Time to start learning

The pace of change in technology is only likely to accelerate in coming years, and the technology we use in farming in ten years may be very different than what is now available. It is tempting to just “wait and see” what evolves, but advanced agricultural businesses choose to embrace technologies that can deliver concrete benefits and which align with their goals. Consider ways that technology can help you achieve more effective communication, improved understanding of business metrics, and the ability to assess and respond to change.

What technologies are you embracing in your business, and why?

Is connectivity interrupting your agricultural vision?

The last few years (even the last few months) have seen a surge in organisations exciting us about what farmers will do with sensors and mobile devices. We’ll be able to collect data with drones flying above our fields and tiny devices scattered across our farms. We’ll collect data with smart ear-tags and boluses in livestock, and we’ll crunch it all with powerful cloud-based analytics and smartphones will be a convenient way to tap into these analytics so that we can make decisions on the move.

That’s the promise.

Given time, much of this is achievable, though being savvy business people, farmers will only adopt technologies that deliver value or reduce risk. Even if the technology does provide significant value the may also be a challenge in connectivity; how do all these devices connect together and to the cloud?

I was thinking about this just last week, when I was demonstrating livestock management software to a group of farmers. It was a wintery day with snow on the hills, and we were standing out in the sheep yards, protected from the sun (but not the wind) by the steel roof.

We had just demonstrated how trivially easy it was to capture information about animals and synchronise it onto a smartphone, and now we were going to show the same information synchronised to the cloud, through a web browser and a mobile data connection.

We pressed “refresh” and we waited – and waited.

Did I mention that I had plenty of time to think?

Of course, eventually the page loaded and the demonstration carried on, reasonably successfully too. The farmers of course took the opportunity to point out that their remote yards would have no coverage at all!

We face a variety of different connectivity challenges in rural environments:

  • Low bandwidth and high latency connections;
  • Connectivity black-spots and intermittent coverage; and
  • Areas with no connection at all.

Low bandwidth and high latency

In many cases, rural networks (fixed or mobile) provide significantly lower bandwidth than those in the cities, or they may have significantly higher latency (time to respond). For some applications, this doesn’t matter at all, but for others it may be a show stopper.

I watched a drone software manufacturer demonstrate the features of a very smart unmanned aerial vehicle (UAV). On-board software used the GPS to navigate the device over a flight plan we outlined using Google Maps. A multi-spectral camera captured high resolution images at 1cm per pixel, and the device transmitted the images to the cloud while it was still flying. That’s pretty incredible!

Transmitting images as they are captured is a great innovation. The intensive processing power required to stitch images together and analyse them is provided by a remote data centre and applied to data arriving from drones around the world. It also frees the operator from messing about with memory cards or thumb drives and processing software.

This works just great on the 4G mobile networks in the Salinas Valley, California. Not so much in rural Wales where the mobile networks drop back to GSM or EDGE, with significantly lower throughput. Still, there may be opportunities to address this. If the drone has sufficient memory, it could cache images until bandwidth improves, or when it returns to a connected base. Alternatively, flying at 400ft above the terrain may provide better coverage than we experience on the ground.

High latency rural connections provide a different challenge. We typically experience an increase in latency with satellite connections, as signal travels through a far-distant satellite, to and from ground stations. Satellite connections can still be very high bandwidth (high data throughput), but have a measurable delay, as you’ll notice if you use a satellite based phone service – “over”.

This latency won’t affect most applications to any noticeable extent, but consider the case of a livestock auction happening at a remote location. Smart auction software allows internet bids to be synchronised with the bids happening on site, right down to subscribers hearing and seeing the auction in real time. A delay of one or two seconds in this case becomes significant, so auction planning and operation needs to take this into account.

Mobility “black spots”

Much of the rural countryside lacks mobile coverage. Hills, trees, and buildings can all affect local reception, especially as distance from a cell tower increases.

This limits our dependence on these networks for “life and death” situations. There’s a reason why Search and Rescue services encourage hikers and hunters to use Personal Locator Beacons (PLBs) rather than rely on mobile phones when in remote areas, and the same should apply to tools that are triggered when an ATV rolls on the farm – the vehicle could well be in a gully outside of cellular network coverage.

We should also plan on intermittent connectivity when developing apps for the farming sector. Requiring an internet connection to start or use an application will limit its use to favourable environments close to home where that coverage exists. An answer of course is smart synchronisation of relevant data, when devices come back into network coverage.

I was quite confident demonstrating our livestock management application remotely for example, as the list of animals had already been stored by the mobile device and most importantly the application was designed to work disconnected and sync later. If there was no coverage, the new data I had captured would be sent to the server when a connection became available.

Bringing your own network

There are always spots with no connectivity: sheep yards in a valley between steep hills; installing nitrate sensors in a creek in a ravine; even equipment in the shadow of a large steel building. What options are there if we must install our technologies where there is no connection?

It turns out there are a range of options, but the need to be technology savvy and the cost of equipment rises in these cases.

  • Consumer networking such as Wi-Fi and Zigbee mesh networks operate at 2.4GHz, providing line-of-sight connectivity over relatively short distances. Trees, hills, and even buildings can block these signals. Wi-Fi’s real strength is that it is a relatively low-cost and easily deployed option. It’s less suitable where extreme battery life and low power is a requirement.
  • There are also specialist wireless networks. These often use lower frequency radio transmissions, operating at 433, 868, or 900-921 MHz. They require a larger antenna, but these frequencies can travel longer distances without increasing the power requirement, and may be less prone to blockage by trees. Some of these networks transmit point-to-point between sender and receiver, and others form a mesh communicating between multiple devices.
  • New standards are evolving for low-power, long-range wireless networks such as the LoRaWAN and SigFox networks. Telecommunications providers are starting to adopt these standards to provide specialist connectivity for “Internet of Things” devices and remote locations. Where these are used at low frequencies in rural networks, they may well provide better coverage for monitoring devices in previously poor locations.
  • Finally, we might consider using mobile phones to “collect and deliver” data from monitoring devices in places where there are no coverage, but where coverage spots are regularly visited or passed. A remote sensor with some memory and Bluetooth Low Energy capability might deliver summary information to your smartphone when you pass by. This is the same approach that is used by activity sensors you wear on your arm – data is captured, and delivered to your phone at intervals.

So there are possible solutions even to those areas with currently poor or non-existent coverage – but many of these require some skills, choices or trade-offs, and often a higher investment cost, and this makes adoption of technology more challenging. Companies developing new products can’t cater for every possible network technology, and may end up integrating a just subset of connectivity options.

Our team at Rezare Systems often discusses these challenges and the evolving set of solutions, because these will enable some of the software solutions we “brain-storm”. We’re not telecommunication providers or hardware developers, but we are very interested in how connectivity influences adoption of new technology and our ability to capture and leverage data.

Have you had experience with connectivity issues in rural applications? Can you share advice for farmers and rural professionals about some of the suggestions above and their alternatives? Let us know your thoughts.

What do consumers know about your supply chain?

Consumers. A jaded and cynical bunch. I include myself in that statement.

Just last weekend, a lovely salesperson was extolling the praises of a new smoothie product (“would you like to try it sir, it’s packed with fruit”), while I was remembering comments from my children about the level of sugar in smoothies and trying to see what was on the ingredients panel.

Studies by the Hartman Group would suggest that consumers are interested in more than just what a product’s packaging looks like, instead wanting to know:

  • What ingredients are in the food or beverage product (64%);
  • How a company treats animals used in its products (44%); and
  • From where a company sources its ingredients (43%).

Of course, that’s not to say we are always completely logical and analytical. When I buy Bella Pane bread at our local farmers’ market, I don’t ask to see the ingredients list, or the best-before date, or ask when it was made. Probably Mike has already told me he got up at 3am to bake the day’s bread, but even if he hasn’t done so, I gain a level of confidence and trust from his local proximity, previous discussions, and the farmers’ market brand story.

That level of trust and confidence in product quality, source, and ingredients is what supports positioning of premium food products. A large North American corporate recently discovered that promising “Food with Integrity” was only a start, and those promises needed to be backed with processes and checks to maintain confidence in their products.

I’ve spent a while recently considering how the information we collect on farm can support the broader story about premium protein products. The Hartman Group research would tell us that consumers in the US are interested in:

  • Hormone free (52%);
  • Free of antibiotics (49%);
  • Artificial (48%);
  • GMO-free (41%); and
  • Organic (31%).

When it comes to animal welfare consumers want to know that companies avoid inhumane treatment of animals – and while they may not know the details of what that means, the proportion of people who care is rising:

  • Other animals are not harmed in capture/raising (e.g. bycatch) (68%);
  • Animals are raised in as natural environment as possible (65%);
  • Animals are not used for product safety testing (65%);
  • Animals are not given hormones or antibiotics (63%);
  • Company supports animal welfare causes/organisations (51%);
  • No animals at all used in products (45%); and
  • Animals fed only organic food (33%).

We know that products and processes that meet these criteria – and more importantly, have a compelling story in these areas – may command a premium in the market, and are in a position to build stronger, more defensible brands.

Consumers expect products and brands to live up to the brand story they are told. When lack of integrity in process or supply chain is exposed, consumers act angrily, as though we have been “tricked” (read Seth Godin’s “All Marketers are Liars” to learn more of how this works).

For that reason, any claims we make about our agricultural products having green origins or being “very pure indeed” need to be backed up by guides, processes and records that demonstrate our commitment to those brand values. Claims of greenness or purity are potentially for naught if we don’t have both safeguards and evidence in place.

Hence the importance of Farm Assurance or Good Agricultural Practice programmes, and the need for audits and for simple to use, on-farm record keeping tools that back up the story. We’re working on some of the latter with our partners. It’s hard work, because farmers are busy people with limited finance. In order for supply programmes to really deliver the benefits promised by the brand, I think we need to do two key things:

Link the activities to the brand story

Make sure everyone who has a role in the supply chain understands how their role contributes to the brand and to the consumer experience. Spell out how actions on farm impact the supply chain: safety, provenance, and in-market claims. Ensure staff know the risks to the business if product integrity fails.

Make it easier to comply than not

Most audit schemes today run on paper – recording pages in a paper book or filling in forms. For practical reasons, these are filled in at the farm office, and often updated just before the auditor arrives. We remove a substantial barrier if it is easy to capture information in the field rather than spending evenings in the office. Reusing information captured for farm assurance records to provide insights for farm management aligns goals and makes adoption more likely.

Your thoughts?

Consumer expectations have been changing over the last decade. Our supply chains and production systems are evolving to meet those expectations. This will require a greater commitment from us all to transparency and integrity, making sure what we do lines up with what we claim.

Do you manage a supply programme, or participate as a farmer, grower or processor? We’re interested in your thoughts. Drop me a note in the comments, or contact me directly.

Getting the most from farm data

Increasing pressure from commodity returns, input costs and environmental compliance means that farming today relies on consistent, quality decision-making. Good information, viewed properly to gain insights, is the life-blood of great farm decisions.

Unfortunately, the most useful data is often hardest to collect and interpret. Pasture information relies on pasture walks (or drives); stock condition must be assessed manually or using advanced equipment; and even understanding growth rates of cattle or sheep requires pulling them off feed and into yards where the risk of transferring disease increases.

Many advisors from fertiliser and feed planning to finance and animal health now have tools that help with visualising outcomes and supporting decisions. In turn, these tools are also hungry for data – sometimes detailed and sometimes high-level farm information. Some farmers tell me they feel every second person up their driveway needs to ask “twenty questions”.

So how can we satisfy our craving for more and better data, without turning farmers into field technicians or survey gurus?

Start with making better use of the data we have

This might include the farmer’s own records in their tool of choice – whether that’s a feed planning tool, paddock recording system, or their financial management system (which often capture product quantities and inventory as well as sale and purchase records). At the moment this existing data is in silos – unable to be accessed because it is locked away, or perhaps in a different format.

Where forward-looking software vendors have made some data available, it is often unable to be directly applied to answer other questions – at least without a human to interpret. Take the example of one tool asking information about calving dates and peak milking numbers, while another asks for monthly cows in milk. With experience and farm system knowledge, a human can readily translate one from the other – but these inferences are hard to automate.

The Farm Data Standards are the New Zealand industry’s approach to getting a common vocabulary, so that our computer systems will be able to meaningfully re-use data. This vocabulary is supported by the Data Linker (a DairyNZ and Red Meat Profit Partnership project), creating standard protocols so that software tools can share farm data through APIs, with explicit farmer permission. Organisations in the Data Linker early-adopter group are building streamlined processes so farmers can re-use their data with little or no overhead.

Towards more automated collection

The “Internet of Things” (IoT) promises to connect sensors and measurement devices from the farm to farm software and databases, making the most of recent advances in consumer electronics to reduce the cost of the electronics, enhance reliability and improve battery life.

IoT devices now available include remote monitoring and alerts for your water supply, pumps and tanks, as well as devices monitoring the state and efficacy of electric fences and effluent spreaders. There have been electronic solutions in this space for quite some time, but improved mobile and on-farm wireless networks, along with smaller and lower-cost electronics, are now making them more attractive.

Coming IoT devices may monitor water quality in real time, assess pasture cover, assist with matching dams and progeny, or with diagnosing animal health challenges. A key for farmers will be ensuring that they can access this data and re-use it for a wider range of purposes where it makes sense.

Filling in the gaps with remote sensing

Lately I’ve been privileged to meet farmers and technology companies in the United States and Australia, where broad-acre cropping of corn, soybeans, and wheat are the predominant farming practice. Farmers are starting to make great use of multispectral and hyperspectral imagery regularly captured from aircraft, low-earth orbit satellites, and even drones (though the range of most drones is too short for larger farms).

Image analysis from these platforms has been around for a long time now (using normalised vegetation difference index or NVDI, for example), but instead of just displaying images and leaving the farmer to guess what is going on, companies are now applying machine learning to correlate the patterns in the images with known crop issues and yields. For large enterprises, this remote sensing data “fills in the gap” between what the farmer observes by walking in the fields, and the wider enterprise. Hyperspectral imaging that captures additional wavelengths will support more sophisticated analysis, and I look forward to seeing some new crop-specific analyses in the future.

Weather and climate data from MetService and NIWA can also be considered remote sensing data to can support decision making, even for those without their own on-farm weather station. The NIWA Virtual Climate Station Network (VCSN) provides a grid of historic climate data and weather data across New Zealand, and that data is combined with soil drainage and fertility information in the Pasture Growth Forecaster. Other countries provide similar climate data services.

A word to the wise regarding Pasture Growth Forecaster: free regional averages are just that – averages over a broad area and a range of soils. You’ll get better mileage by paying the trivial amount each month to get a custom forecast based on your location and soils.

Bringing it all together

I’ve painted a bright picture of how the data available from a number of sources – existing databases and suppliers or customers, small in-field devices connected with the Internet of Things, and remote sensing data – could reduce the overhead that currently puts many farmers off collecting data.

The challenge for farmers and their service providers is now to bring those assorted pieces of data together to provide information and insight for better decisions.

For service providers (including software developers such as Rezare Systems) that means lifting our sights from simplistic tools that regurgitate input data in pretty graphs, to providing predictions, visualisation, and insights that support decisions which matter to farmers. For farmers, that will mean grasping technologies that show potential to address future farming needs, and challenging vendors to make systems as open, connected, and useful as possible.


Building APIs for your agricultural product

Providers of products, services, or data in the primary production sector are under pressure from their users and partners. Everyone wants to exchange useful pieces of information (data) rather than forcing farmers and customers to read information from one application and retype it into another. They not only expect data to flow, but applications to collaborate, providing benefits of automation, efficiency, and the whole being greater than the sum of the parts.

In this cloud computing and internet-connected world, we do that through web service interfaces called APIs. Application Programming Interfaces have been around for a long time (you may recall people using APIs to automate Excel and Word).

Today’s APIs are implemented through web servers, although unlike web sites, web service APIs are designed for other computers to consume, so there are no pretty user interfaces, just code. Web servers can scale to handle high loads and use secure protocols to protect data, so web service APIs are a logical step to make data available securely and to handle unpredictable demand from other systems.

Grasping the benefits

There are benefits to be gained over time by improving how your product, service, or data system integrates:

  • Increasing connectivity with the other systems that your customers or suppliers use, enabling them to make better informed decisions, improve product quality or productivity;
  • Leveraging the investment you have already made in capturing and cleansing data, turning that “hygiene” activity into something that adds value to your supply chain;
  • Reducing duplication of data entry with its attendant increase in errors and latency; and
  • Opening the door to two-way flow of information with your suppliers, collaboration partners, and others in the supply chain.

Challenges for existing systems

Connecting legacy applications through web services can be challenging if those applications were designed as standalone tools, client-server platforms, or even delivered through “green screen” terminal interfaces. These applications won’t have been designed for connection to the web.

  • Modern web applications are “stateless”, freeing the server from maintaining the state of a transaction or workflow between calls from the client. This allows web applications to scale to thousands or millions of users. In web applications, the web browser itself may hold state as it provides the user experience, but this may not be possible for other systems connecting to our API. Careful design of these APIs is needed to ensure they don’t just reflect the flow of the original legacy application.
  • Legacy applications may not support the technical features that would allow us to connect them directly to a web server. Often only file input and output or “scraping” data from an ANSI terminal screen are available, so that intermediate technologies are needed to collate and make data available.
  • One of the largest challenges we’ve found is that engineers who maintain legacy applications may have not been exposed to web technologies. They are great developers and grasp the concepts, but there is a significant learning curve to the new platforms, and these developers are already committed to maintaining and enhancing the core system.

Some approaches to consider

We’ve assisted our customers to connect their existing agricultural applications to the world of REST web services, and it is worth sharing some of what we have learned.

Can we access the database? Sometimes we’ve been fortunate to have access to the underlying SQL database used by the application, which makes it feasible to expose data through an API. Care needs to be taken though:

  • The security model of the application might not be fully represented through the database, so you may need to reproduce this or take other steps to avoid exposing data to which users should not have access.
  • Inserting or editing data may not be possible, or may require significant effort to enforce the business rules for that data. If you’re lucky, stored procedures or other server-side mechanisms might be in place to help with this.
  • The legacy database might not be able to provide the up-time or performance needed to handle hits from external systems. We worked with one database where the overall system performance was already marginal and adding more load would have brought the server to its knees.

Can an intermediate database help? With less accessible data storage, and systems where adding significant load to the existing database would be a problem, we have sometimes been able to use file export or database sync to take a copy of the data. We then use that database to provide data through the API. There are some additional advantages to this approach:

  • We can re-map the data to a simpler data model, or one which better reflects how people will use the data;
  • We can implement ways to secure the data that align with external access rules;

However, we also need to remember that this method will introduce latency – transactions that take place in the legacy system won’t immediately be available through the API (depending on the synchronisation method used).

Adding or updating data through a web service API takes some planning too (whether you use an intermediate database or not). Approaches might include:

  • Posting or updating the data into the intermediate database, and then making the “synchronisation” process two-way to push data back to the legacy system; or
  • Accepting the data and turning this into a queued or batch data load job into the legacy database. In this case, the post or update request would return an “in progress” status and the calling system would not be able to access that data until the loading task had completed.

Inserting or updating data via a web services API raises other design questions, such as how to handle rejected data and other exceptions. Is there a roll-back mechanism or a way of discarding changes that were initially accepted but then found wanting? How do we notify the application that supplied the data if the API call was asynchronous? There are approaches to all these, but it certainly takes experience and makes for some robust design discussions!

Rezare Systems routinely undertakes this sort of work for its customers, and right now we’re working through a lot of these challenges with the Data Linker project. I welcome your thoughts, suggestions, or questions – do get in touch.

Agricultural Tech Reloaded

MobileTECH 2016 in Rotorua, New Zealand was a showcase and discussion forum for the adoption of smart technology in primary industries: agriculture, horticulture, and forestry. 300 attendees from a broad spread of primary sector organisations and technology companies spent two days discussing the application of all things sensor, wireless, and cloud.

I thought it worth summarising four major themes from my perspective.

Read More

Four ways to make your farming app engaging

So you’ve decided to build a mobile app for your rural customers or users. You have spent time on the overall value proposition, deciding how your app will deliver enough value to your users that they will spend the requisite time – and money – to use your service. No doubt you’re also modelling the likely market penetration and adoption curves, because you want to make sure you can get a return for all the investment in software development, testing, and marketing.

You know that up to 20% of mobile apps are downloaded, used once, and then never touched again (perhaps uninstalled when the user runs out of space). How can you avoid your app ending up in that category? Better than that, can you provide an experience sufficiently valuable and engaging that users tell their friends?

Read More