"Modern slavery is a hidden crime" declares publicity for the 'Modern Slavery Garden' a gold medal winner this week at the RHS Chelsea Flower Show. Congratulations to Juliet Sargeant, the garden's creator, and the first black designer and winner in the show's 104-year history. The garden, a symbolic depiction of forced servitude behind closed doors and hopes for a brighter future, has garnered widespread coverage and drawn attention the creation of the Modern Slavery Act. The 2015 legislation aims to criminalise the perpetrators and beneficiaries of organised exploitation, believed to affect 27 million people globally and 13,000 in the UK, with an estimated $150 billion created annually in illegal profits through slave labour.

The group behind the garden, whose committee includes the Bishop of London, prominent Christians Frank Field MP, Baroness Cox, and representatives of the A21 campaign, has also launched the #Askthequestion campaign. They request that consumers tweet their favourite brands and ask whether their products are free from slavery. The desired outcome is that brands feel the pressure of public scrutiny and investigate their suppliers, subcontractors, and everyone one involved in the creation of their products, and report on this in compliance with the new law. This, the campaign hopes, will be the start of significant change. "We have the power to change this and make slavery history" declares the Modern Slavery Garden website. Who is 'we'? Apparently those with purchasing power. "Just photo [sic] a product, tag the company that made it and post to #askthequestion" states the campaign.

To do this, what's required is a smartphone, or digital camera and computer, an active social media presence, and the means to purchase the goods being photographed, or at least to download a picture, which suggests a campaign aimed at a relatively wealthy and connected constituency targeting the same demographic. Charlie Hart, one of the team, says they are a "bunch of gardeners" with a keen interest in drawing attention to the continuing existence of slavery, and after completing their ambitious Chelsea project they hope the campaign will see an organic take-up and increase awareness of the breadth of the modern slavery problem.

All this is positive attention highlighting a terrible global situation – and well done to a team of enthusiasts who chose to use their own passion to create a platform for conversation – but it scratches the surface. If anyone chooses to claim the title abolitionist, and link themselves to the legacy of reformer William Wilberforce, more must be done.

In the age of instant information is anything really hidden? Only by choice. The limits of social circles, and choice of media can hide things. They can be hidden if they are far removed from experiences of work, and the wider world. Having worked for companies of the size (£36 million annual turnover and over) now required to declare where slavery exists in their supply chains, I'm aware of the lack of interest and priority most give this. Unless setting out with a social agenda, companies exist to make money. They will only be breaking the new law if they don't declare, not if they declare and don't act. The new 'kite mark' will reward good intentions not actual change. All of this suggests very little shift is expected or desired in the systems and behaviour of consumers, producers, and legislators.

Slavery is a crime of power and inequality, something the global economy depends on, however unpalatable that reality. While targeting larger businesses for supply chain accountability is a promising move, the reality of modern day slavery – and the application of the law thus far – is often on a much smaller scale. The first conviction for the new crime of 'Domestic Servitude' was for an abusive man subjecting his wife to a tormented existence upon her arrival in the UK. Other investigations have related to a carwash in Kent, a Yorkshire bed factory and a family of labourers in Wales. Blessed relief for those freed from horror with opportunity to live different lives, but a drop in the ocean of a global problem.

To be a 'modern day abolitionist' requires more than a campaign, hashtag or good intentions. To genuinely want to see the end of slavery means an up-ending of the current inequalities that have brought them about in the first place and a willingness to see and live differently. It is easy to fall into 'othering' while discussing this, holding to a simplified narrative of good and evil, abolitionist and slave holder, vulnerable or exploited and exploiter. To assume rescue is the answer while hoping the wider problems will resolve themselves if enough risk to reputation is created. Cindy Berman of the Ethical Trade Initiative, that works with retailers and producers to bring about change, says "The criminal justice system is a hammer. The problem is, modern slavery is not a nail" and different thinking about causes is needed.

Additionally, labour groups highlight that the new laws offer no compensation to freed slaves, make access to legal aid fraught and tricky, and have not abolished tied visas that restrict workers to the employers who brought them into the UK. Additionally, a slave must have been 'travelled' by the slave keeper rather than making their own way into a situation, reflecting the Government's immigration concerns.

Is the end result literally to remove individuals from the control of other individuals on a case by case basis? Not to change the system? Is capitalism all good if 'good' people are in control? Should consumers continue to use brands and luxury goods having asked the question, and transfer responsibility to the makers? Is the overall answer to criminalise the actions of one or two but not the system that drives the need? The Government intends that company's declarations will allow consumers to make "more informed decisions at the checkout" but not to consider greater change. Every person with spending power participates in exploitation, knowingly or not (see the recent prawn crisis, for example). Slavery has a history almost as long as humanity. The wider public now believes it to be a bad thing, yet benefits from it daily. Slavery is often the poor exploiting the even-more-poor. The caricatures of the evil overlord counting piles of money, or the olden days plantation owner lording it over subhuman property, don't apply.

Slavery exists in products not because the majority of people are determined to be evil but because they want to make money – the same as the brands do. A poll produced ahead of the new law suggested some customers would swap brands to avoid using slavery-produced products – including 75 per cent over aged 35 – yet how many are prepared to do the research? And how is a younger generation, who have access to more info than any other before them, inspired to act when they don't currently feel compelled to change their habits?

With slavery and exploitation a known part of the global flower trade, how can the enthusiasm and success of the Modern Slavery Garden team spark a wider rethink and approach to worldwide problem? Let's not just stop at #askthequestion.

Vicky Walker is a writer, among other things. Her book 'Do I have to be good all the time?' about the meaning of life, love and awkward moments is available now. Follow her on Twitter.