Mobile Life

firmware testing

It doesn’t matter whether you’re salivating for Ice Cream Sandwich, or it’s iOS that sets your pulse racing – we know how much it matters to our customers to be running the latest version of software available on their handset and to get it as soon as possible.

It can be a complicated process though, as Vodafone’s Chris May explains.

“The first step in the process is for manufacturers to decide which devices will receive or need an update. In the case of a major firmware upgrade – such as Android devices moving from Gingerbread to Ice Cream Sandwich – it’s largely dependent on the experience that the combination of the new software and the hardware capabilities of the device can offer customers. That call is made by the manufacturer in the first instance”

“It’s the first time we’ve seen Ice Cream Sandwich on a Huawei”

With the decision made to updgrade the firmware of a device – as Huawei plans to do soon with the G300, moving from Gingerbread to Ice Cream Sandwich – the manufacturer can start to develop their device’s version once they receive the software from Google, in the case of Android.

It’s at this point that the Vodafone-specific elements of the software build are implemented.


touchscreen futureThe secrets of tomorrow’s touchscreens
The tech behind the display’s one thing, but what about the future of our phones’ shiny screens? Click here for a look at tomorrow”s phone tech.


“We work through our global team to ensure that the features and settings we want to provide our customers with are incorporated into the build of the update. In some instances we add branding and services to a device that we believe will benefit our customers – be that music, apps or even remote diagnostics capabilities. In other instances, we need to pre-configure services such as voicemail or network settings. This means the customer gets the best possible experience when they fire up their newly updated device for the first time.”

Testing times

Once Vodafone engineers get their hands on an upgrade they start a rigorous series of tests, with the aim of being first to deliver the highest quality product to customers – but there can be several iterations of the software before it’s made available for customers to download.

“Firmware testing can typically take anything from one day to one week, depending on our previous experience with an individual manufacturer, and the complexity of the upgrade itself,” May says, “Security releases and bug fixes are usually quickest to test, but platform upgrades with new features take longer”

Software tests“We conduct smart testing,” IT Technical Manager Kevin Ralph explains. “If it’s a manufacturer we know well, we might conduct more limited tests – focusing on how new features perform. Assuming that goes smoothly,  this process can result in the software making it back to the manufacturer pretty quickly.”

The team also studies the manufacturer release notes that outline any tweaks to the device.

“We test data, the operation of the apps, battery life, music services and so on – but if it’s a device we’ve seen before we wouldn’t go back and test the antenna again, unless there’s been a specific issue identified with it.”

In the case of the Huawei G300 Ice Cream Sandwich upgrade, however, engineers are conducting the full range of tests, including network bearer tests:

“We’re doing nearly all the testing we’d do for the launch of a new device. It’s the first time we’ve seen Ice Cream Sandwich on a Huawei, and the first time we’ve seen them do this kind of upgrade – so it’s important we test everything and make sure the user will get the best experience.”

Ralph added that the Huawei has a relatively straightforward manufacturer overlay on Ice Cream Sandwich, which makes it less complicated than more customised devices.

In addition to testing the platform on a live network, engineers also look at the proposed delivery mechanism – and whether it can be supplied FOTA (Firmware over the air).

“A platform upgrade is a big file, so we suggest to customers that they upgrade via WIFI. In fact, where we can we like to ensure there’s an alert included for users to that effect during the update process itself” Ralph said.

According to Chris May, operators are seeing a definite trend towards larger upgrades, and multi-stage testing cycles:

“Typically software and firmware now goes through a two stage testing cycle. After conducting a first round of tests we report back to suppliers with our recommendations for any improvements – and then we would expect to see that upgrade again before we could be confident that it meets the highest standards for our customers.”

Although the Vodafone team is responsible for testing all of the software updates released to our customers, a given software’s release date is determined by a number of other factors.

“We discuss an outline of delivery times with the manufacturers at an early stage to ensure that customers don’t get too many upgrades in too short a period – but once we have tested and approved the upgrade it goes back to the manufacturer to review before being sent for approval by Google, in the case of Android,”

“In terms of testing, it’s our job to ensure that we turn it round as quickly as possible, while still ensuring the best customer experience of the final software,” May says. “We work very closely with our manufacturing partners throughout this process as you’d expect.

“Once our testing’s completed we’re keen for the update to get through the remaining steps in the process smoothly and quickly so it’s in our customers’ hands sooner rather than later. In many cases, our customers are among the first to receive software updates, and we’re proud that’s the case”

Back in time: The journey from 2G to 4G is an amazing one. Find out how we went from the first text message to where we are today, and where we’ll be tomorrow, too. 

Share this story

Add comment

  • Nabs

    What happens with iOS updates that might be different from Android?

    • Hi Nabs,

      Each operating system or manufacturer has its own approach.  In the case of Android, there are many different manufacturers, customisations and versions of software in circulation at any one time. By contrast, iOS updates are usually made available to a smaller range of devices simultaneously. However, the same principle of our being able to test the software before it is made available to our customers still applies, as it does with all other operating systems.



  • Tim Jones

    Hi Guys,
    Many thanks for a really interesting article. I am sure many people will want you to expand this more with regard to iOS Firmware 😉

    As you will be aware the VodafoneUK eForum operate the DLP, (Device Loan Programme) for top contributers.
    We have reviewed devices in the past for the Vodafone Social Blog Team, so you know how intensive our Customer oriented testing can be!
    It would be great to utilize the DLP with Firmware testing to get an immediate Customer feedback on Firmware updates prior to Public release. The infrastructure of lending devices is already in place!
    This would make an awesome story for the Blog team to follow from inception to reality.

    Perhaps you could mention this to the Testing Team and get back to me?

    This would be a great way for Vodafone and their Customers to work together to provide a better Service. 🙂

    • Hi Tim,

      We’re really glad you liked it! You’ve made an interesting point here. Because of the really tight timescales involved between our testing team receiving the software and it being made available for distribution by the manufacturer, it’s not possible for us to involve our super user community at this stage in the process.

      That said, every update that our customers receive is tested rigorously by our team and we do pay close attention to feedback we receive from customers as they download the latest update. That means in the rare event that there is an unforeseen issue that comes to light as the update starts to roll out, we can identify and work with our manufacturing partners to resolve it as quickly as possible.

      We’ll certainly mention the idea to our testing team though 🙂


      • Tim Jones

         Hi Dan,

        Thank you for your reply. I’m pleased that you will mention the suggestion to the Testing Team.

        VodafoneUK are very fortunate to have a Community that are so willing to be involved in the improvement of Service. It would be great to see something evolve and flourish from this article.


  • Graham Hale

    Do we get to Know the results of the testing ?Do Fix roms get rolled out as soon as the manufacture need them to ?
    Why is is a Manufacture rolls out a Rom and Vodafone don’t eg the Galaxy S this has a new look and apps and Vodafone users are left wanting ?
    Who tests Branded apps on previous android devices they don’t work?
    why do Vodafone apps run in the Back ground ?
    What do the apps do ?
    Why do Vodafone apps use data running in the back Ground ?
    Why can we not Un install these apps some device will alow this eg Sony on others were stuck with them and have to disable them if we have Ice cream sandwich
    Why is there no one to contact on Rom problems or to get information on Roms?

    This Blog is Good and has been needed but it does not apply to all manufactures 
    Samsung have rolled out a Fix for the S3 and it has new features but no one has told the S3 users what this fix is and what’s it for and why Vodafone users will not be getting or or when oh this has been out for a Month now well out of the Testing Time scale

    • Hi Graham,

      As you know, all of the latest information we have on specific device updates is made available through the eForum – it’s also one of the places you can get support around specific device issues you may be experiencing. The eForum team works really closely with our testing team, which means they always have the most up to date details.

      In terms of how and why Vodafone customises the software on our devices, you’ll see in the piece above that there is a variety of different approaches, ranging from making it easy to connect to the network when the device is first used, to giving access to services we believe will be useful for the majority of our customers.

      In saying that, it’s not just about services. It’s also about support. Pre-installing features like Help Login enables our Tech Team to resolve a greater number of issues remotely, which means customers can often hold on to their phone instead of needing to send it in for repair.

      From a technical perspective, many of these features are baked-in to the core software (ROM) on the device, which means removing them is not possible through the approved update cycle that’s used to deliver fully tested and certified software to our customers.

      We recognise that some people will choose not to use the services that are pre-installed on a phone – be that operator, manufacturer or software vendor driven content – so the size of pre-installed files is kept to a minimum and customers have the flexibility to position them on the device as they choose.


  • Looking forward to the ICS G300 update even more now! Is it confirmed that it`ll have openGL enabled?

    • Nathan Hulse

      I think the above post is referring to 2D OpenGL acceleration. The source is there for all to use on the Qualcomm site, yet even Huawei have not added it yet. This modification would be considered an enhancement by every customer.

    • Glad to hear it! The best place to get info on specific updates is the eForum. 

  • A E

    But a B927 ROM is already out which gives us pretty much the same thing.
    Plus, while you lot were slowly trying to fix the speaker issue with B885, people were running B888.

    Answer how you are so far behind.

  • I’m assuming this question refers to the Huawei G300? 🙂 The best place to find out about specific software enquiries is the eForum and they’ll give you all the info they can. I do know they would only have cause to test them if the were being rolled out to Vodafone customers by a manufacturer though. 

  • It may have been said many times before, but just to be sure: I believe there are many people (like me) that are not as interested in manufacturers (bad) or network operator (worse) overlays to Android than they are to get the latest stock Android upgrades asap.
    That aspect alone makes me think that a ‘pure Google’ device like the Galaxy Nexus is much more attractive than any Samsung/HTC or even Vodafone-ized devices: Only 1 company to test it properly, and only 1 company to delay the process.

  • when will ICS be officially released for xperia S?

    • Hi Scott,

      That’s a question you’d need to pick up with Sony, I think. Vodafone doesn’t sell the Xperia S so we won’t have info on the update plans that Sony has. 


  • Ian Dunsmore

    How will I get the update now that it’s available?

    • Hi Ian,

      You can visit our eForum for all the current information we have on schedules for the availability of specific updates. However, I can tell you the ICS update for the G300 hasn’t been released yet as it’s currently being tested.


  • Hi Stefan,

    You’re quite right to point out that different people want different things from their smartphone, which is why it’s great that there are so many choices out there, both in terms of hardware and software. Just out of interest, is the Galaxy Nexus your first device from the Nexus line-up or have you had others? 


  • A great insight into how things happen – but misses the part where you then include a load of bloat-ware into your ROMs before pushing to your customers and then prevent them from removing any of it without rooting. 

    For example why is the 360 Shop running on in memory on my device and in the 18months of owning my Desire HD I have never used it once. 

    Sure ship with these apps – but don’t lock them to the device so I can’t remove them with a simple un-install.

  • OK but let’s talk numbers.   What stage is the Xperia Mini Pro at?  I know people on other networks already have Ice-Cream on the mini pro so does that mean we are just waiting for Vodafone to start to customise it or finish customising it or are we in the test stage?

    • Hi Robin – the guys on our eForum always have all the latest info on specific device updates. http://forum.vodafone.co.uk 

    • A E

       The Xperia Mini Pro is Sony Erricson device. Unless you want to head over to XDA to install a Custom ROM, the device will never get ICS, because Sony and Erricson don’t make phones together anymore.

  • Nabs

    I guess there is some reason Huawei haven’t included it. Might be worth dropping their support team a call 

  • manamanamana123

    can you download ics on a huawei ascend g300 from Norway?

  • galaxy note.

    hi. just found this forum. I’m trailing behind most people as I’m still on Gingerbread 2.1 so any update will be greatly received. Though sooner rather that later please.

Stress-free summer holidays: 2016’s best apps, tips and essential downloads

roaming smartphone thumb
Mobile Life
Keeping kids happy on your summer holidays is no mean feat, but the right tech can, as always, help take the stress out of travelling abroad. Here's Ellie Allen's essential app guide...

Thursday 28 July 2016

Itsy-bitsy, teeny-weeny, Internet of Things bikini: Check out Vodafone's Smart Swimwear concept

SmartSummer thumb
Internet of Things
Mobile Life
Think the Internet of Things is just for appliances around the home? Think again! As our smart bikini concept shows, IoT is set to land on just about everything...

Wednesday 27 July 2016

Appy Travels: 2016's best holiday apps

Mobile Life
All set for your summer holiday? Don't board that plane until you've downloaded our resident travel guru's favourite trip-planning and vacation-curating apps...

Tuesday 26 July 2016