  IntroductionIn 2022, the North East LEP completed an evaluation of the Made Smarter Adoption North East (MSANE) programme, which explored the design and delivery of the support, uptake and effectiveness of the support, and the broader effects on the ecosystem. The evaluation also sought to understand the programme’s scalability, ‘what worked well’ and ‘what worked less well’ and best practice learning from benchmark programmes. Leadership and training programmeavailable to businesses through workshops and one-to-one coaching Networking and peer learning opportunitiesavailable to supported businesses Access to grant fundingto implement a technology project Support to make productivityimprovements. Design and delivery of MSANEResourcingThe MSANE launched in 2021, and was resourced via £600,000 of funding from the broader (£8m) UK government Made Smarter Adoption programme allocation, plus top-up additional funding of £120,000 to deliver eight Peer Networks. The pilot has been overseen by the North East Growth Hub and Tees Valley Growth Hub, and focuses on growing awareness of, interest in, and ability to pursue adoption of digital technology among manufacturing SMEs. Type(s) of support offeredProgramme funding has been allocated to deliver a leadership and training programme (made up of workshops and one-to-one coaching), networking and peer learning opportunities, alongside access to grant funding to implement a technology project. MSANE is part of the broader Made Smarter Adoption programme[1], which comprises regional programmes in the North West, Yorkshire and Humber, and West Midlands. Each regional programme follows a similar structure. The delivery model of MSANE is broadly in line with similar programmes that support technological adoption in traditional sectors elsewhere in Europe (however these programmes were nationally oriented in terms of scale). Overall, programmes that focus on digitalisation in industry have targeted four core functions: i) Advice, ii) Loans or grants, iii) Networking, and iv) Creation and sharing of best practice guidance. While the generation of best practice is not covered within MSANE, the programme covers each of the other aspects, and focuses on facilitating learning and relationships between participants rather than solely financial provision. Networking and knowledge exchange elements are highlighted in the literature as effective practice[2] including its role in assisting businesses by either providing advice to fill knowledge gaps, or signposting to other potential sources of knowledge. Literature highlights ‘essential’ areas of practice seen in MSANE including clarity of what is available and on offer, and simple, low-barrier to access[3] Further, in advisory based business support programmes hands on approaches where there is a focus on advisors generating trusted relationships often leads to better outcomes[4]
 Business characteristicsThe primary audience for the programme was manufacturing SMEs that could benefit from exploring and adopting digital technologies to support productivity improvements. The programme targeted engagement with 100 businesses, offering expert support to a sub-set of 80 businesses. Businesses were identified by delivery partners. Recruitment of participants required a specific approach, as the ideal participants are ‘less digitally active’ and smaller than comparators for other programmes such as HPSU. This changes the way in which participants should be engaged – while approaches such as LinkedIn may still be effective, it was thought likely that potential participants may engage less frequently with such platforms. This resulted in a lower conversion rate than for example HPSU, stated in consultation as around 8%, presenting further challenges. To address this, outreach support was secured through a dedicated call centre contract (which was largely successful in engagement). In consultation with programme leads, the number of businesses supported was stated as 76 across the North East and Tees Valley (95% of the overarching programme target). Of 44 businesses recorded between August 2021 and March 2022 for the North East LEP area, 28 received high-intensity assists (64%) and 16 received medium-intensity assists (36%).  majority received high-intensity assistscompared to other forms of support
82%
operating in the manufacturing sectorwith remaining businesses operating in knowledge intensive business services
78%
micro or small businessesreporting under 50 employees in North East LEP area largely based in Sunderland and Durhamcompared to other areas within the North East LEP area Effectiveness of the MSANEBusiness benefits and performanceConsulted businesses reported motivations for joining the programme, including upgrading existing systems, exploration of existing ideas around digitalisation (including finding good practice examples), networking opportunities, and access to grant funding. The extent to which these motivations have been realised, plus broader insight into business benefits and impacts was assessed via consultation with stakeholders and businesses. Consulted stakeholders received positive feedback from businesses regarding the implementation of technology projects, knowledge gained from peer interactions, and increased awareness of the need to digitise to keep up with market demands. All business interviewees stated the programme had helped advance their business in line with their motivations. Specific examples of technology projects discussed by interviewees include:
Furthermore, interviewed businesses suggested that their progress would have been slower without the programme or would not have happened at all. The sample of interviewed businesses represents 18% of all companies supported by MSANE, and this sample could be divided into two types: those who participated aiming to learn more about opportunities and good practice, and those with a pre-existing plan for which they wanted advice and funding. Those who reported having a specific aim tended to report having acquired a specific type of system or installed ether own machinery, while the individuals with fewer pre-existing notions reported both a better view of opportunities and potential concrete improvements (e.g. software). Both types of business leaders perceived the programme to be worthwhile and useful, and it is clear that they have been helped to explore and adopt new technologies, with concrete examples of business impact. A qualitative counterfactual (contribution analysis) made up of a selection of hypotheses deemed relevant to the programme was completed. The results of this analysis are positive related to upskilled employees, readiness and ability of businesses to assess and adopt digital technologies, and use of technology to streamline processes. The as yet un-supported hypothesis relates to increased turnover, which is largely due to the fact that only a short period of time has elapsed since the pilot. Otherwise, the unique nature of the programme in the North East makes the assessment of the contributions of the programme to these outcomes and impacts easier. Effects on business outcomesConsulted businesses were able to list numerous beneficial outcomes and impacts including (though not limited to) enhanced networks, improved knowledge around digitisation and a better sense of surrounding opportunities, but none of these were deemed to be outside of what would be expected. Effects on policy and the broader ecosystemConsulted stakeholders mentioned that MSANE enabled increased awareness of other parts of the support landscape, such as university provision. It was noted by stakeholders that businesses often feel overwhelmed by the scale and complexity of the support sector and where to go, and are often sceptical of approaching support ‘cold’, due to the possibility of engaging with ‘sales representatives’ that are trying to sell a product. One stakeholder specifically mentioned the complementarity of other support such as the Peer Networks programme that was folded into MSANE provision to provide a more rounded experience. A smaller number of consulted businesses discussed the difficulty of the support landscape. One business reported that joining one programme often has implications for being able to join others, resulting in time spent ‘weighing up’ options and the relevance of available support. MSANE was seen as a provider of independent advice, simplifying this process, and helping the business to better understand their needs, steering them in the right direction. Our contribution analysis sets out a mixed view of the impact of the programme on the ecosystem. Much of this is arguably related to the small scale of the pilot, which makes any sizeable system-level impacts difficult to realise. Benchmarks of programme performanceIt is difficult to produce precise benchmarking of the MSANE programme due to the relatively early stage this evaluation has been conducted post-intervention. However, an overarching qualitative assessment suggests that the observable results of the programme (and early indications of business impact) are in line with the benefits seen among comparator programmes. Our review of comparator programmes in Germany, Austria, and Spain, as well as pan-European interventions, shows that programmes to support digital technology adoption are designed to deliver similar outcomes to MSANE, and that support provision is broadly comparable. Each of these comparators are also at a relatively early stage and thus have not yet been fully evaluated, further mitigating direct comparison via the benchmarking framework. However, high-level insight into performance of similar interventions is available via a 2018 Policy Links study carried for Innovate UK, which developed case studies of 212 digitalisation projects in manufacturing supported by more than 70 national digital adoption initiatives[1] This review found the most common reported improvements from digitisation are increases in process efficiency (30% of cases), reduction of labour costs (16%), and reduction in defects and errors (10%). However, there is very little information on how outputs varied by the type of support received within the publicly available report. Our review of other regional Made Smarter programmes found little information available on what has been achieved to date through the Yorkshire and Humber and West Midlands programmes, but the longer running Made Smarter North West: Technology Adoption (MSANW) Pilot has reported encouraging results among the 155 technology projects supported through the programme. Headline findings of an evaluation conducted by Steer finds that over 80% of SMEs working with the programme have seen a boost in productivity, and more than 25% reduced their carbon emissions. Projections estimate this will lead to a £118m increase in gross GVA for the region within three years. Future ConsiderationsScalabilityConsulted stakeholders suggested that an expanded three-year roll-out of MSANE will follow the pilot and were unanimously positive about the scalability of the programme, including both supply and demand factors. However, it was stressed that this would rely on increased awareness and understanding of the programme, and the availability of digital experts. The few businesses who had a view of the scalability saw no barriers, and instead encouraged further efforts to raise awareness. Areas that worked well and suggested improvementsWhen asked to consider what aspects of the programme had worked well in delivering results, interviewees mentioned four examples:
When asked to consider what aspects of the programme could be improved in future, interviewed stakeholders and businesses mentioned the following:
 Identification of best practiceLooking at more general research around business support programmes, and the literature as a whole across several areas relating to digitalisation, there are several things that are identified as lead to successful support programmes[1]
 Overarching evaluation of the North East Growth Hub and its activitiesIntroducing the overarching evaluation, headline findings and key recommendations.  Evaluation of the North East Growth HubInterested in the learning and outputs from the North East Growth Hub evaluation.  High Potential EvaluationTo find out more about the learnings from the High Potential Start Up programme evaluation.  North East Growth HubTo explore what business support is currently available. |
