Quantcast
Channel: Linaro Blog – Linaro
Viewing all 179 articles
Browse latest View live

U-Boot on ARM32, AArch64 and beyond

$
0
0

U-Boot became the de facto bootloader on most ARM systems during the early 2000s. It grew out of an earlier flora of smaller and custom boot loaders such as RedBoot and Open Handhelds ARM Bootloader. Currently the main alternatives are the Little Kernel bootloader, which has been used by Qualcomm and Google for a series of Android devices, and the UEFI-compliant Tianocore (also known as EDK II) bootloader.

What is the best bootloader to use for any one system is a subject of debate. There have been pushes to different “there can be only one” approaches, but the recent consensus is to “use the right tool for the job”, while people may have differing opinions on what the right tool is.

Boot Chain
All SoCs have some way to bootstrap their CPU(s) to execute code on cold start. On an older ARM32 system, the execution is usually started in an on-chip ROM, which in turn continue execution either in NOR flash (memory-mapped flash memory) or by initializing the main RAM (which is normally not accessible at boot) and loading a proper boot loader from a NAND flash or eMMC or SD card.

Sometimes several steps need to be performed to boot a system, and as some code may need to execute from on-chip memory or locked down cache until the RAM is initialized, initial steps can be very small boot stages (programs).

Eventually a fairly large program single-threaded program is loaded into memory, and its task is to load and execute the final operating system from images (binary objects, files) stored on some media. For simplicity, this program is usually executed from 1-to-1-mapped physical memory. This program may also have the ability to reformat and install new images on the system.

This program is referred to as the boot loader. The stages up until this program is loaded is handled by ARM Trusted Firmware on the ARM reference designs for AArch64.

The boot loader will typically be a bit interactive (has a prompt) and support booting the final operating system from hard disk, memory card, flash memory, ethernet connection, USB cable, or even through light morse code from an IrDA sensor. It places the final operating system image in memory, passes some information to it and kicks off execution at the start of executable memory.

From this point, the operating system needs to set up virtual memory, caches and everything else needed to get the system into full-flight mode.

Chain of Trust
If a chain of trust shall be preserved across these stages, the first point of execution needs to be trusted and contain routines for checking validity of the next executable program all the way. This is usually achieved using public key cryptography, where a public key is stored in the ROM (or similar location inside the device) and binaries to be executed need to be signed by the secret key corresponding to that public key. This way the device will not contain any secret keys. Sometimes a certificate chain is used to distribute the signing authorization.

Initial U-Boot AArch64 Support
The AArch64 (ARM64) support for U-Boot was pioneered by Scott Wood, David Feng and York Sun from Freescale in 2013 to support their LS2085 platforms. Leo Yan from Marvell joined the efforts, and thanks to these people U-Boot can start and boot Linux on a range of ARMv8/AArch64 systems.

ARM Fastmodel Support
Freescale’s submissions included fastmodel support, a specific customization known as Foundation model or simply FVP. This is a cycle-exact AArch64 emulator made by ARM Ltd, which behaves akin to a Versatile Express reference board, just with the difference that the whole system is emulated in software.

In order to load binary images into the emulated memory, so-called semihosting is used. This is basically a way for the code running on the emulator to talk directly to the emulator, i.e. for it to be aware that it is not running on real hardware. By issuing a parametrized HLT instruction, the code running in the model can ask for services, such as to retrieve files into the memory, from the emulator.

When I started working on AArch64 support for U-Boot I augmented this code a bit so that we now have a command called smhload that will load a file into the emulated memory akin to how files are loaded from flash memory or over Ethernet+TFTP.

By working on the Foundation model, I could verify that execution and interactive prompt was working, and I could continue with support for real reference hardware.

ARM Juno Development System Support
Freescale’s attempt had been focused around emulated reference hardware and later their own hardware. When I started working on AArch64 the scope was on the 64 Bit Juno ARM Development Platform. The idea was to showcase U-Boot on this real hardware as a reference point for the rest of the ARM vendor ecosystem. If we could get U-Boot working nicely on Juno, we could provide a trusted starting point for others.

First we had to make Juno start the compiled U-Boot. ARM recommend that U-Boot is started from the ARM Trusted Firmware, which is essentially the ROM for the Juno. The trusted firmware performs the boot chain as described above in several stages or Boot Levels called BL1, BL2, BL3-1, BL3-2 BL3-3. I only needed to consider myself with the last boot level, BL3-3, which is the level containing a “real” bootloader binary. In the examples, BL3-3 was Tianocore UEFI. By compiling U-Boot to address 0xe0000000 and replacing UEFI with the resulting binary, U-Boot was executed by the ARM Trusted Firmware.

At first the system would not boot at all – the Juno went catatonic. By instrumenting U-Boot with a low-level UART print hack to push strings to the console before initializing the rest of U-Boot, I could determine the cause: the MPIDR (Multi-Processor ID register) had totally different meaning and contents on a multi-cluster machine. The U-Boot code was adapted for a single cluster of symmetric CPUs, not for multiple clusters of CPUs, such as the cluster of two Cortex-A57s and four Cortex-A53s found on the Juno.

Freescale’s system had the ROM or similar mechanism enter U-Boot from both CPUs, and when it reached U-Boot all slave CPUs were immediately dispatched to a spin table while execution of the single-threaded U-Boot should continue on the primary CPU. However the branch_if_slave assembler macro would think all CPUs on the system were secondary CPUs.

Since the Juno board was only initiating execution of the boot loader on the primary CPU, this problem was solved with a patch making U-Boot assume single entrance (i.e. only one CPU will execute it) and after this we got all the way to prompt. A special configuration symbol, ARMV8_MULTIENTRY was created for systems such as Freescale to select. This way single-entrance was made the norm.

Now U-Boot was working to prompt at Juno hardware, so I could test loading a kernel by compiling in Y-modem binary loading support and uploading a kernel Image file and a device tree to the memory and start execution using Y-modem and boot it. It worked fine. A patch for initial Juno support was submitted upstream and merged.

Uploading a big kernel and initramfs over the serial port at 115200 baud was quite tiresome, so I immediately started to get U-Boot to load kernels over the ethernet port, resulting in a patch supporting SMSC9118 ethernet booting. This was it is possible to quickly boot a kernel using ethernet and TFTP.

It was now quick and efficient to develop Linux using U-boot, especially if you compile in a boot script into the ethernet/TFTP boot so that all you really need to do it reset the machine and it would immediate download a new kernel from the TFTP server and run it.

However it is nice to be able to flash a kernel and a filesystem into the on-board flash memory in the Juno and use that to just boot the machine, especially for demos and similar where you want to prepare the machine and just use it. Thus I also added flash support to the Juno, the tricky part being a patch to handle the AFS partitions in the flash – this was a new ARM-specific flash image format that relies in footers in the end of the last erase block of the flash. After adding this, I could make a patch making this the default boot method for the Juno, so the boot chain was self-contained on the device.

Future Directions
We now have pieced together a system that will start U-Boot from ARM Trusted Firmware and then have U-Boot load the Linux kernel and a device tree and start it. Are there problems remaining?

  • One of the big outstanding issues are those where things are fragile because memory references need be hard-coded in U-Boot or ARM Trusted Firmware. For example U-Boot currently assumes that ARM TF will use 16MB of the DRAM memory. If the ARM TF change things around and use more or less memory, U-Boot needs to be reconfigured and recompiled. U-Boot on the other hand, will then pass whatever knowledge it has about the memory to the Linux kernel by augmenting the device tree. So if ARM TF could communicate the memory available to U-Boot and the OS this would be great.
  • U-Boot relies on prior boot stages such as ARM Trusted Firmware to install PSCI handlers, while on ARMv7 this was usually done by augmenting U-Boot to do the same. Letting U-Boot install PSCI handlers is a bit bogus, since it is a piece of resident code left in memory after U-Boot has executed and not really “boot loader” code. U-Boot was augmented to compile these into a special memory area, copy them there and leave them around for the operating system to use later. Still there are people who might like to do this on ARMv8 U-Boot, especially those not using ARM Trusted Firmware.
  • People apparently toy with the idea of booting U-Boot on bare metal, using a very small or no ROM nor ARM Trusted Firmware, letting U-Boot just execute immediately on the system. As U-Boot relies on something else to set up main memory and providing PSCI, this currently does not work. Doing this would require U-Boot to initialize memory and install PSCI handlers. It would also need to be small enough to execute from on-chip RAM.
  • Chain of trust booting with signed boot levels, signed U-Boot and a signed kernel image and a signed device tree, making an example of a totally locked-down system. The Flattened Image Tree (FIT) supported by U-Boot is likely the best way forward here, but requires U-Boot to access public key infrastructure to verify images unless you want to compile the public key directly into U-Boot, which is often not a good idea.
  • Fastboot – the Android boot protocol used by the Little Kernel, exists in U-Boot but has not been tested or verified. It can use USB or Ethernet alike.
  • More hardware support – such as booting from the USB stick or MMC/SD card found in the Juno board. This was not covered by the experimental port.

Linaro Digital Home Group at SFO15

$
0
0

The Linaro Digital Home Group (LHG) will once again be very active at Linaro Connect San Francisco 2015 (SFO15) and has scheduled a wide array of interesting events, presentations, and demos.

Linaro Connect is a great time for all the LHG engineers to get together to showcase to the open source community what we’ve been up to over the last six months.  On Tuesday September 22nd there will be several sessions dedicated to Digital Home topics as well as demos later in the week that are aligned with the overall ‘Security’ theme of SFO15.

LHG will have the following presentations on Tuesday:

Title: SFO15-201: Boot Architecture for RDK

Abstract: A uniform boot sequence allows RDK systems to have predictable start-up behavior. RDK would like to use a two-level boot architecture, where the primary bootloader is device specific and initializes system registers, memory controller, and hardware resources, and the secondary bootloader implements more complex boot requirements like code download, system disaster recovery and firmware validation.

Title: SFO15-205: OP-TEE Content Decryption with Microsoft PlayReady on ARM TrustZone

Abstract: This presentation gives an overview of how various components of set-top software are integrated to provide a W3C EME solution employing a commercial DRM integrated with an open source TEE running on ARM TrustZone.

Title: SFO15-209: Cisco Connected Life IoT Gateway

Abstract: Experiences productizing the gateway (from hobby to product) and the challenges for IoT deployment scenarios: Home security, Home automation, Home healthcare, and Enterprise IoT (smart manufacturing).

In keeping with the security theme for the week, LHG will have a special event dedicated to “Security in the set­top box” on Wednesday Sept. 23rd. The event is entitled: Expanding security choices: DRM and CA Interoperability.  The event will feature the CTO of Verimatrix, Petr Peterka, who will give a presentation discussing Enhancing Operability for Robust Revenue Security. This presentation will be followed by a panel discussion with industry leaders discussing the challenges and opportunities in the evolving set­top security space.

Finally at the traditional Linaro Connect Demo­ Friday, LHG will be highlighting the following demos:

  • ­Microsoft PlayReady Integrated with OPTEE
  • ­Linaro Clear Key solution on HiKey Board
  • ­Qt Apps with Wayland for the RDK

Those attending Linaro Connect are welcome to come to any and all of the LHG sessions and events.  If you are not able to attend you can visit the Linaro Connect SFO15 page to view recorded sessions and get access to select materials.  Hope to see you there.

Going to SFO15? You will need to know Pathable

$
0
0

Linaro Connect SFO15 is just around the corner and we are expecting another great event.  There will be daily keynotes given by several industry leaders, numerous interesting sessions that attendees will want to participate in, afternoon hacking and a lot of meetings.  So how do you keep track of everything happening each day?  You will need to logon to the very easy to use Pathable event schedule system.  

Pathable allows attendees to see the daily schedule of all activities, select just the ones they would like to attend, schedule meetings, have discussions with other attendees and receive important notices of any changes or additions to the schedule.  It is also where all the content for each session will be posted first, so any links to presentations or videos will be posted under each session as soon as they are available.  

This site will be the place to help you get the most out of your time at Linaro Connect SFO15 so it is really important that you:

  1. Go to the site and login prior to the event
  2. Set-up your profile and become familiar with the site and how it works
  3. There will be a “mobile app” available for you to install from the app store to access the community.  This will be coming shortly.  We will let attendees know as soon as this is available.

Below are some steps to help get you started:

Step 1:  Go to the pathable site:  https://sfo15.pathable.com/

Step 2:  Login – If you have registered for Linaro Connect SFO15 you should have received an email from pathable after 10 Aug with your login details. This email was entitled: “Please join Linaro Connect San Francisco 2015″. If you cannot find this email please send an email to: connect@linaro.org

Step 3:  Once you get logged in – You will want to update your profile page so that other attendees know who you are, as well as set-up your password to something you can remember easily.  

Step 4:  Add Tags to your profile to identify teams you work with or areas you are interested in.  You can see a list of the current tags being used by going to:  Attendees>Tags (in the left hand drop down), but you can add your own as well.  When people perform a “search” in the attendees or members list, they will be able to choose from the networking interests and tags you and the rest of the community have added to their profiles.

Step 5:  Review the current schedule https://sfo15.pathable.com/meetings  and add sessions you would like to attend to your schedule https://sfo15.pathable.com/meetings/my.  

  • Click the Schedule tab
  • Hover your mouse over the session description box that pops up that you would like to select and click the “Add” button that appears.
  • If you are on a tablet computer or wish to read more before making your choice, select the name of the session you are interested in. Then click the “Add to My Agenda” button to add the session to your schedule.
  • You can then go to the My Schedule page to see your agenda for each day

Step 6:  If you need to have any meetings with other attendees you can also easily set those up through Pathable.  

  • Visit the profile page of the person you would like to meet
  • Click the Schedule a Meeting option
  • If you would like to add additional people to the meeting, type their names into the “Attendees” field
  • You will see the schedule and free-busy time for you and the invitees on the right. Use your mouse to select a time that is mutually available.
  • Select or enter a Location for your meeting*
  • Type “Subject” and “Description” for the meeting to give the invitees background on why you would like to meet
  • Click the Send Invitation button.

​Your recipients will receive an email notification for the meeting with the option to accept or decline your request. You will receive an email notification when they have responded.

*Most meeting rooms are books however, there are many areas around the hotel to meet.  There is the option to have your attendees meet at the registration desk or put in your own location.  If you need a specific room please see the registration desk.

For more help with Pathable:
These are the basics for Pathable however, there is much more you can do with the tool; including messaging other attendees, having a group chat, adding documents for a sessions etc.  To get more details you can visit the Pathable Knowledge Base to learn more.  http://support.pathable.com/knowledgebase

Linaro 15.08 Release Available for Download

$
0
0
“Tell everyone what you want to do and someone will want to help you do it.” ~ W. Clement Stone

Linaro 15.08  release is now available for download.  See the detailed highlights of this release to get an overview of what has been accomplished by the Working Groups, Landing Teams and Platform Teams. The release details are linked from the Details column for each released artifact on the release information:

We encourage everybody to use the 15.08 release.

This post includes links to more information and instructions for using the images. The download links for all images and components are available on our downloads page:

USING THE ANDROID-BASED IMAGES

The Android-based images come in three parts: system, userdata and boot. These need to be combined to form a complete Android install. For an explanation of how to do this please see:

If you are interested in getting the source and building these images yourself please see the following pages:

USING THE UBUNTU-BASED IMAGES

The Ubuntu-based images consist of two parts. The first part is a hardware pack, which can be found under the hwpacks directory and contains hardware specific packages (such as the kernel and bootloader). The second part is the rootfs, which is combined with the hardware pack to create a complete image. For more information on how to create an image please see:

USING THE OPEN EMBEDDED-BASED IMAGES

With the Linaro provided downloads and with ARM’s Fast Models virtual platform, you may boot a virtual ARMv8 system and run 64-bit binaries.  For more information please see:

GETTING INVOLVED

More information on Linaro can be found on our websites:

Also subscribe to the important Linaro mailing lists and join our IRC channels to stay on top of Linaro developments:

KNOWN ISSUES WITH THIS RELEASE

For any errata issues, please see:

http://wiki.linaro.org/Cycles/1508/Release#Known_Issues

Bug reports for this release should be filed in Launchpad against the individual packages that are affected. If a suitable package cannot be identified, feel free to assign them to:

UPCOMING LINARO CONNECT EVENTS: LINARO CONNECT SAN FRANCISCO 2015

Linaro Connect San Francisco 2015 will be held September 21-25, 2015.  More information on this event can be found at: http://connect.linaro.org/sfo15/

What to expect at SFO15 next week

$
0
0

Welcome to Linaro Connect San Francisco 2015!

We are returning to the Hyatt Regency Hotel in Burlingame, CA which is conveniently located next to the San Francisco airport.  Hence, the shortcut reference of SFO15.  

Why the same venue?  It could be because we were hoping to overlap with the YaoiCon[1] convention as we did last year.  Trust me, that is definitely not the reason although the convention was a topic of some interesting hallway discussions.  As with most Connects, this venue offers a combination of convenience to a major airport as well as access to a cosmopolitan city with great people, restaurants, nightlife and attractions.  In addition, the San Francisco Bay Area is a world center of innovation and is home (headquarters or major facility) for many of our member companies.  

This Connect will be similar in format to HKG15 (translation, Linaro Connect Hong Kong 2015).  The overall theme for the week will be security and there will be daily themes for each of the four segment groups.  Simon Segars (ARM CEO) and George Grey (Linaro CEO) will be kicking off the week on Monday with opening keynotes.  Expect to hear a glimpse of things to come for ARM and the ARM Ecosystem.

There will be daily keynote speakers on Tuesday through Friday presenting topics relevant to the segment group theme of the day.  Keynote speakers include Neil Trevett (VP Mobile Ecosystems, Nvidia), John Simmons (Media Platform Architect, Microsoft), Suresh Gopalakrishnan (VP & GM – Server, AMD), Dave Neary (SDN/NFV Community Strategy, Red Hat) and Karen Sandler (Executive Director, Software Freedom Conservancy).  We have a surprise second keynote speaker on Thursday.  A surprise because, as I write this blog, we haven’t finalized who it will be yet.  Not to worry, it will be someone and something of interest.  On Friday there will be a panel discussion on security.  The panel will tell us what we should know, think we know or are afraid to know about this timely and critical topic.

SFO15 has special significance to those who have been with Linaro since the beginning (2010).  It is the five-year anniversary of Linaro and we are planning to have some fun during the Gala dinner on Thursday night at the Computer History Museum[2].  Some of you may remember we visited the museum at a previous Linaro Connect several years ago.  However, this visit will be a bit different.  How different?  Sign-up for the Gala dinner to find out.  There will be a limited number of seats so I recommend signing up early.

And, of course, it will be a week full of interesting sessions, demos, hacking (the good kind), meetings, discussions, good food and good people.

Speaking of good people, what will Andrew Wafaa do to amaze us this Connect?  

For full details on SFO15 schedule click here.  This Connect is bigger than ever, so big we had to close registration early.

See you all in San Francisco!

 

[1] https://en.wikipedia.org/wiki/Yaoi-Con, YaoiCon 2014

[2] http://www.computerhistory.org/

Energy Aware Scheduling (EAS) progress update

$
0
0

Authors:  Ian Rickards (ARM),  Amit Kucheria (Linaro)

Today, power management on Linux is implemented by different subsystems that work in a largely un-coordinated manner. This makes platform adaptation difficult and tuning complex. ARM and Linaro are jointly developing “Energy Aware Scheduling”, a technique that improves power management on Linux by making it more central and easier to tune.  This will improve mainline Linux support for advanced multicore SoC’s that power current and future mobile devices and other consumer products.

The existing Linux ‘Completely Fair Scheduler’ has a throughput based policy.  For example, if you have a new task and an idle cpu, then the scheduler will always put the new task on the idle cpu.  However, this may not be the best decision for lowest energy usage.  EAS is designed to implement energy saving without affecting performance.

The Energy Aware Scheduling project consists of a number of component tasks:

EAS task image

The goal is to introduce generic energy-awareness in upstream Linux:

  1. Using a clean, generic design to support a broad range of CPU topologies.
  2. Based on scientific, measured energy model data rather than magic tunables.
  3. Providing a high-quality baseline solution that can be used as-is, or extended as needed.
  4. Designed-for-mainline => reducing software maintenance costs.

EAS will unify 3 separate frameworks in the Linux kernel that are currently only loosely connected:

  • Linux scheduler (Completely Fair Scheduler – CFS)
  • Linux cpuidle
  • Linux cpufreq

These existing frameworks have their own policy mechanisms that make decisions independently. Our previous blog post covered the limitations of this approach.

The optimal solution is to fully integrate these functions into the Linux scheduler itself, with sufficient information to enable the most energy-efficient scheduling decisions to be made.

A typical ARM multi-core SoC would have the following voltage and frequency domains:

ARM voltage EAS blog

Ideally, each cluster will operate at its own separate independent frequency and voltage.  By lowering the voltage and frequency, there is a substantial power saving.  This allows the per-cluster power/performance to be accurately controlled, and tailored to the workload being executed.

A generic energy model based approach is expected to support a broad range of current and future CPU topologies, including SMP, multi-cluster SMP (e.g. 8-core Cortex-A53 products), as well as traditional ARM big.LITTLE.

Since the original discussions started on the Linux Kernel Mailing List in 2013, there has been significant progress recently:

AES blog image 3

Scheduler idle-state awareness

Engineer:  Nicolas Pitre, Linaro [Merged Sep-2014, in Linux 3.18 and later]

The sched-idle enhancement makes the scheduler aware of the idle state of the CPU’s.  When waking up a cpu it will now always pick the CPU in shallowest idle-state, minimizing wake-up time and energy.

In the example below, a new task needs to wake up, but it will not fit on CPU#0 because the current operating point is almost fully utilized.  With sched-idle integrated, the new task always gets placed on CPU #1 since it is in the shallowest idle state (WFI), and the other cluster remains in C2 shutdown.  This is the lowest energy and fastest response option.

EAS blog 4

DVFS (cpufreq) improvements

Current situation with DVFS support in Linux

The existing cpufreq implementation is an extension to the Linux kernel, which uses a sampling-based approach to consider cpu time in idle along with some heuristics to control the CPU Operating Performance Point (OPP).  There are a number of disadvantages to this approach:

  1. Sampling based governors are slow to respond and hard to tune.
  2. Sampling too fast: OPP changes for small utilization spikes.
  3. Sampling too slow: Sudden burst of utilization might not get the necessary OPP change in time – reaction time might be poor.
  4. Only aware of the overall CPU loading and is not aware of task migration.

EAS blog 5

New scheduler-driven DVFS (sched-DVFS)

Engineers:  Mike Turquette, Linaro/Baylibre [latest PATCH v3, June-2015]

With scheduler task utilization tracking, a feature that the mainline kernel already supports, any OPP transition required will happen immediately based on the stored tracked load of the task.

EAS blog 6

With sched-cpufreq, when the new task is placed on CPU#1, the cpu capacity for the little cluster changes immediately.  This uses the history of the task, which is stored internally as part of the CFS scheduler in the kernel.  This is a good approximation for many tasks which have consistent cpu load behavior.

Foundations – Frequency and capacity invariant load tracking

Engineers:  Morten Rasmussen/Dietmar Eggemann, ARM

The “Per-Entity Load Tracking” (PELT) framework in the Linux kernel determines the load of a task by looking at the utilization of cpus.  The existing design of PELT tracks the CPU utilization but does not accurately track the load on different CPUs at different frequencies or with different performance per MHz.  ARM has built on the recent July-2015 rewrite of PELT from Yuyang Du to add frequency and microarchitecture support:

https://lkml.org/lkml/2015/7/15/159 – PELT rewrite (Yuyang Du, Intel corp.)
https://lkml.org/lkml/2015/8/14/296 – Frequency and microarchitecture invariance for PELT  (ARM)

Capacity
This is a measure of the processing capability of a cpu.  ARM patches include enhancements for capacity to be extended with additional scaling for microarchitecture and current operating frequency. The cpu capacity at different operating points is based on measuring some standard benchmark metric ,e.g. “sysbench”

Utilization
Traditionally the utilization has been related to the running time.  ARM foundational patches extend this to accommodate the frequency & performance of the cpu.

Existing utilization calculation

EAS 8

New utilization calculation takes into account frequency and microarchitecture

EAS image 9

Energy model

Engineer:  Morten Rasmussen, ARM [latest RFCv5, July-2015]

The EAS energy model is the final piece which enables the CFS with energy-aware task scheduling.  It allows the kernel to decide at run-time which scheduling decisions are the best ones for lowest energy usage. The Energy-Aware policy is to always pick the CPU with sufficient spare capacity and smallest energy impact.

This also removes the magic tunables in some of the power management frameworks at present – you actually have to look into the code to understand what these magic tunables do.  For example, consider the big.LITTLE HMP thresholds, the scheduler tunables, and even the interactive governor tunables (used in product but didn’t make it to mainline)

EAS image 10

The platform energy model is an accurate baseline model of the dynamic and static power used by the CPUs in the system.

Typical big.LITTLE CPU power/performance curves

EAS image 11For each CPU, the energy model contains the following information

EAS blog image 12

We are discussing the best ways to express this energy model with the open source community. One option that is being considered is using a Device Tree

Options for placing a waking task
As seen in the diagram below, a newly waking task can sensibly be placed on either of the two CPUs – CPU#1 or CPU#3.   With the current mainline scheduler, either CPU#1 or CPU#3 could be chosen. 

EAS image 14

EAS considers the energy costs of the two options:

CPU#1: operating point must be moved up for both CPU#0 and CPU#1

CPU#3: no operating point change, but higher power used as per Power/Performance graph below

EAS image 15

Based on the above, EAS will probably choose CPU#1 because the small additional energy cost of increasing the OPP of CPU#0 (and CPU#1 by implication – since both CPUs are in the same frequency domain in this example) is not significant compared with the better power efficiency of running the task on CPU#1 instead of CPU#3.  The key foundational pieces are understanding the intensity of the task (done by PELT with frequency & microarchitecture invariance).

EAS doesn’t evaluate all the possible options. That can introduce performance hits in key scheduler pathways. Instead,  EAS narrows down the search space to:

  • CPU the task ran on last time.
  • CPU chosen by a simple heuristic which works out where the task fits best.

Based on the energy model, EAS evaluates which of these two options is the most energy efficient.


SchedTune

Engineer:  Patrick Bellasi, ARM [posted August-2015]

The ‘interactive governor’ appeared on Android in 2010, and it has proved to be a very popular solution for maximizing battery life whilst providing a high operating point suitable for interactive tasks. However, the interactive governor was not merged into the mainline Linux kernel. There is considerable interest in having a frequency boost capability available in mainline Linux as part of cpufreq (and potentially EAS in future).

There has been a repeated demand to have a single, simple tunable ‘knob’ that permits the selection of energy efficient operation at one end and high performance operation at the other end. With sched-DVFS and EAS in place, the stage is set for implementing such a central tunable. ARM’s proposal for this tunable is called SchedTune.

SchedTune adds an additional ‘margin’ into the tracked load from PELT. Sched-DVFS and EAS then use this ‘boosted’ tracked load when selecting operating points as usual. The magnitude of the margin is controlled by a single user-space facing tunable.

EAS image 16

If the task appears to be bigger, the allocated MHz from cpufreq/sched-cpufreq will be higher.  Also, on a big.LITTLE system, it is more likely to be placed onto a big cpu. This simple technique permits the selection of a suitable power/performance point that provides the best interactive response for the system.

 

Tooling & Analysis

ARM & Linaro have been working on implementing opensource test and analysis tools, most of which needed to be newly developed for the EAS project.

rt-app/ WorkloadGen  (Linaro)

https://wiki.linaro.org/WorkingGroups/PowerManagement/Resources/Tools/WorkloadGen

Most existing benchmarks run flat-out, and there are few good existing tools to run lower-intensity use cases.

rt-app is a linux command-line tool that creates light intensity workloads, using json files to describe different simulated use-cases. rt-app is already used by the scheduler community.

workload-automation (ARM)

https://github.com/ARM-software/workload-automation

This is a python framework for running standard tests and benchmarks on a target system. It supports:

  • Linux
  • Android (browser and standard benchmarks)
  • ChromeOS (telemetry benchmarks etc)

Kernel ftrace logs are captured from the Linux kernel, and workload-automation integrates with various power measurement tools, e.g. NI DAQ for measuring device power, and ChromeOS servo boards.

TRAPpy (ARM)

https://github.com/ARM-software/trappy

https://github.com/ARM-software/bart

trappy is a python-based visualization tool to help analyze ftrace data generated on a device. It depends on ipython notebook and pandas (python data analysis library), and can be used from a browser to zoom in to analyse scheduler behaviors.

One important feature is it contains an API used for tracking behaviors for thread residency, which allows it to be used as the framework for regression testing for EAS.  ARM has a tool called “BART” – Behavior Analysis Regression Testing which uses this API.

idlestat (Linaro)

https://wiki.linaro.org/WorkingGroups/PowerManagement/Resources/Tools/Idlestat

Idlestat uses kernel frace to monitor and capture C-state and P-state transitions of CPUs over a time interval. Idlestat can also use an energy model for a given platform to help estimate the energy consumption of a given workload.

Idlestat can be used with sample workloads to capture and compare C-state and P-state behaviours in a reproducible manner across kernel versions.

kernelshark (existing)

http://people.redhat.com/srostedt/kernelshark/HTML/

X11/GTK tool used for analysis of ftrace data, useful for detailed scheduler analysis but does not offer the API capability of ‘trappy’ above.

Getting involved with EAS

All the work on EAS is done in the open on mailing lists:

  1. Linux Kernel Mailing List (LKML) for patches and EAS architecture discussions
    (postings on LKML prefixed with “sched:”)
    This is the preferred option as the Linux kernel maintainers will see the questions.
  2. eas-dev mailing lists (http://lists.linaro.org )
    This mailing list is to discuss experimental aspects of EAS developments that are too premature for discussion on LKML

ARM provides a git repo containing the latest EAS patched into a recent Linux kernel

ARM/Linaro are planning an LSK 3.18 backport of EAS (on a separate experimental branch) for availability soon, this will be the best route to Android testing.

ARM and Linaro appreciate any participation in shaping the future direction of EAS, and we particularly welcome testing on a range of platforms including ‘tested-by’ comments on LKML.

Current patchsets for review/testing

Description URL
Scheduler driven DVFS PATCH v3 https://lkml.org/lkml/2015/6/26/620
EAS RFCv5 https://lkml.org/lkml/2015/7/7/754
SchedTune proposal https://lkml.org/lkml/2015/8/19/419
Foundational Patches
(frequency and microarchitecture contribution to capacity/utilization, split out from RFCv5)
https://lkml.org/lkml/2015/8/14/296
Yuyang Du PELT rewrite v10 containing ARM enhancements to utilization calculation  (already queued for merging) https://lkml.org/lkml/2015/7/15/159

Future patches under development

Proposed Patch
big.LITTLE awareness on wakeup path
Further Scheduler driven DVFS  enhancements
SchedTune extension for EAS

Further reading

LWN Article: “Steps toward Power Aware Scheduling”  (25-August-2015)
http://lwn.net/Articles/655479/

LWN article: “Teaching the scheduler about power management” (18-June-2014)
http://lwn.net/Articles/602479/

LWN article: “Power-aware scheduling meets a line in the sand” (5-June-2013)
http://lwn.net/Articles/552885/

Linaro Connect 2015 Kicks Off in San Francisco

$
0
0
SFO15 Group Photo

The largest Linaro Connect to date began today in San Francisco.  Linaro Connect San Francisco 2015 (SFO15) will take place this entire week from September 21-25, 2015.  This marks the fifth anniversary of Linaro Connect and there are many exciting things planned for this week.

The day began with two opening keynotes.  The first was by George Grey, CEO of Linaro.  George began his keynote by welcoming the attendees to the fifth anniversary of Linaro Connect and Linaro.  He discussed some of the highlights over the last five years and what to expect going forward.  He then went into some of the benefits that members get when they join Linaro.  There was an announcement about a new group that is being formed by the end of 2015 in Linaro called, Linaro IoT & Embedded Group (LITE) which will focus on  Cortex-A  Linux devices.    There were also several demonstrations around the 96Boards initiative during the keynote.    Following the Welcome keynote was a keynote by Simon Segars, CEO of ARM who gave a talk on “Collaboration:  Key to Delivery During Market Disruption”

The Monday sessions began after lunch and attendees had many different tracks to choose from including those focused on security which is the overall theme for the week as well as several on community and networking among others.  Below is a list of sessions that have content available.

Session ID Session Title Track URL to Session Information
SFO15-100K1: Welcome Keynote – George Grey Keynote https://sfo15.pathable.com/meetings/302600
SFO15-100K2: Keynote:  Simon Segars, ARM CEO Keynote https://sfo15.pathable.com/meetings/302601
SFO15-102: ODP Project Update Networking https://sfo15.pathable.com/meetings/302651
SFO15-103: Cross-distribution (ARM Linux platform support) Builds and Baselines https://sfo15.pathable.com/meetings/302652
SFO15-105: Core Development Lightning Talks -Kernel, Power Mgt, Security & Virtualization Lightning Talks Core Development https://sfo15.pathable.com/meetings/302654
SFO15-106: LMG Lightning Talks Mobile https://sfo15.pathable.com/meetings/302655
SFO15-107: LAVA Users Forum Validation https://sfo15.pathable.com/meetings/302656

Be sure to check back in tomorrow morning to watch two great keynotes live.  First up, at 8:50am (PST) will be Neil Trevett, Vice President of Mobile Ecosystems at NVIDIA, who will give a keynote titled “Open Standards and Open Source Together – How Khronos APIs Accelerate Fast and Cool Applications”.  You can watch the keynote streamed live here:  https://plus.google.com/u/1/events/c9u3phgkckq0sn8tfearacbpfvo

The second keynote tomorrow will begin at 9:20am (PST) and the speaker will be John Simmons, Media Platform Architect at Microsoft, who will give a keynote on “The Web and Digital Rights Management – the technical solution to the Web-DRM paradox and its disruptive implications”.  You can watch this keynote streamed live here:  https://plus.google.com/u/1/events/cpd8iven7d60tidsvd7ntcnjl4k

Visit http://connect.linaro.org/sfo15/ for updates throughout the week.

Day 2 of Linaro Connect SFO15

$
0
0

Linaro Connect San Francisco 2015 began with another exciting day of announcements and keynotes.  First up was an announcement that two new members have joined Linaro just this week. Acadine Technologies and Tinno Mobile have joined as members of the Linaro Mobile Group (LMG).

Following the announcements, the morning keynotes began with Neil Trevett, Vice President of Mobile Ecosystems at NVIDIA and President of the Khronos Group, of who gave a talk on “Open Standards and Open Source Together – How Khronos APIs Accelerate Fast and Cool Applications“.  The talk covered how over 100 companies cooperate with the Khronos Group to create open, royalty free API standards to access hardware acceleration for heterogeneous parallel computation, 3D graphics and vision processing.

He talked about the latest updates to API standards including OpenGL, OpenCL, OpenVX, and the recently announced Vulkan new generation graphics and compute API. Neil also discussed how cutting-edge APIs enable compelling experiences such as neural-net based driver assistance, virtual and augmented reality, and advanced 3D scene reconstruction on ARM-based devices, and showed a video demo of this. Finally, he talked about how Khronos is embracing and promoting multiple open source projects to boost the ARM developer ecosystem.

The second keynote of the day was given by John Simmons, Media Platform Architect for Microsoft’s Operating System Group.  John’s talk was titled “The Web and Digital Rights Management – the technical solution to the Web-DRM paradox and its disruptive implications”  His talk overview was: “The Internet poses a fascinating technical challenge for commercial media distribution. As noted in a 2003 United Nations

WIPO report, the principle purpose of DRM is to create an element of scarcity on behalf of a rights holder; but doing so raises a fundamental paradox – the business of the publisher lies in providing access rather than preventing it. The report went on to state: “Nevertheless, unless copyright is to be abandoned as a mechanism for trading in intellectual property entirely, it will be essential to find an answer to this paradox.”

The introduction in 2011 of a global standard for DRM-interoperable encoding and the 2012 Microsoft-Netflix-Google proposal for HTML5 Encrypted Media Extensions (EME) provided a partial answer to this paradox. What remained was to extend this capability to open source applications, thus providing a complete answer to the Web-DRM paradox while enabling open, interoperable media applications with access to enhanced content protection.  This talk covered the technical underpinnings of this highly disruptive, strategic inflection point, the interplay between open source and enhanced content protection and the implications for both commercial video and the Internet.

Following the keynotes the attendees were able to choose from several different tracks of sessions.  Below are some of the sessions that have materials available.

Session ID Session Title Track URL to Session Information
SFO15-200K2: Neil Trevett – Open Standards and Open Source Together – How Khronos APIs Accelerate Fast and Cool Applications Keynote https://sfo15.pathable.com/meetings/302829
SFO15-200K3: John Simmons- The Web and Digital Rights Management – the technical solution to the Web-DRM paradox and its disruptive implications Keynote https://sfo15.pathable.com/meetings/302830
SFO15-200: Linux kernel generic TEE driver  Security https://sfo15.pathable.com/meetings/302831
SFO15-201: Boot Architecture for RDK)  Digital Home https://sfo15.pathable.com/meetings/302832
SFO15-202: Towards Multi-Threaded Tiny Code Generator (TCG) in QEMU  Virtualization https://sfo15.pathable.com/meetings/302833
SFO15-205: OP-TEE Content Decryption with Microsoft PlayReady on ARM Security https://sfo15.pathable.com/meetings/302837
SFO15-206: kernelci.org / Kernel Testing BoF  Product https://sfo15.pathable.com/meetings/302838
SFO15-209:  Cisco Connected Life IoT Gateway  Digital Home  https://sfo15.pathable.com/meetings/302841
SFO15-210: Kernel Mainline Status of Mobile Chipsets  Kernel  https://sfo15.pathable.com/meetings/302842

Be sure to check back in tomorrow for the third day of Linaro Connect.  The morning will feature two keynote speakers, first will be Suresh Gopalakrishnan from AMD followed by Tiger Hu from Alibaba who will discuss “Cloud Computing Infrastructure of the DT Era”.  To learn more about the agenda tomorrow please visit: https://sfo15.pathable.com/meetings


Linaro 15.09 Release Available for Download

$
0
0
“Tell everyone what you want to do and someone will want to help you do it.” ~ W. Clement Stone

Linaro 15.09  release is now available for download.  See the detailed highlights of this release to get an overview of what has been accomplished by the Working Groups, Landing Teams and Platform Teams. The release details are linked from the Details column for each released artifact on the release information:

We encourage everybody to use the 15.09 release.

This post includes links to more information and instructions for using the images. The download links for all images and components are available on our downloads page:

USING THE ANDROID-BASED IMAGES

The Android-based images come in three parts: system, userdata and boot. These need to be combined to form a complete Android install. For an explanation of how to do this please see:

If you are interested in getting the source and building these images yourself please see the following pages:

USING THE UBUNTU-BASED IMAGES

The Ubuntu-based images consist of two parts. The first part is a hardware pack, which can be found under the hwpacks directory and contains hardware specific packages (such as the kernel and bootloader). The second part is the rootfs, which is combined with the hardware pack to create a complete image. For more information on how to create an image please see:

USING THE OPEN EMBEDDED-BASED IMAGES

With the Linaro provided downloads and with ARM’s Fast Models virtual platform, you may boot a virtual ARMv8 system and run 64-bit binaries.  For more information please see:

GETTING INVOLVED

More information on Linaro can be found on our websites:

Also subscribe to the important Linaro mailing lists and join our IRC channels to stay on top of Linaro developments:

KNOWN ISSUES WITH THIS RELEASE

For any errata issues, please see:

http://wiki.linaro.org/Cycles/1509/Release#Known_Issues

Bug reports for this release should be filed in Launchpad against the individual packages that are affected. If a suitable package cannot be identified, feel free to assign them to:

 

Day 3 of Linaro Connect SFO15

$
0
0

Day 3 of Linaro Connect San Francisco 2015 was a day dedicated to ARM Server Ecosystem.  The morning began with two keynotes focused on this topic.  After the keynotes there were several sessions held during the day with guest speakers for the LEG server ecosystem day, with topics ranging from CoreOS, LXD Docker & Ubuntu Snappy, ARM64, Docker images for ARM server and Realtime Streaming Analytics.

The first keynote speaker of the day was Dr. Suresh Gopalakrishnan who is the corporate vice president and general manager of AMD’s server business.  Dr. Gopalakrishnan gave a keynote titled “Enabling ARM Server for the Datacenter”.  He discussed how the key to enabling ARM server technology in the datacenter is all about the software.  He then went on to talk about each of the different areas that need to be addressed.

The second keynote of the day was given by Tiger Hu, architect of Alibaba Infrastructure Service.  The title of the keynote was “Cloud Computing Infrastructure of the DT Era”.

Following the keynotes the attendees were able to choose from several different tracks of sessions.  Below are some of the sessions that have materials available.

Session ID Session Title Track URL to Session Information
SFO15-300K2: Suresh Gopalakrishnan, AMD Keynote https://sfo15.pathable.com/meetings/302930
SFO15-300K3: Tiger Hu – Alibaba Keynote https://sfo15.pathable.com/meetings/302931
SFO15-301: Benchmarking Best Practices 101 General https://sfo15.pathable.com/meetings/302933
SFO15-306: Kernel Consolidation 2.0 – Let’s make it happen!  Kernel https://sfo15.pathable.com/meetings/302939
SFO15-307: Advanced Toolchain Usage Part 5  Tools https://sfo15.pathable.com/meetings/302940
SFO15-309: Expanding security choices panel: DRM and CA interoperability Security https://sfo15.pathable.com/meetings/302942
SFO15-310: Advanced Toolchain Usage Part 6  Tools https://sfo15.pathable.com/meetings/302943
SFO15-311: ConfigFS Gadget – An Introduction  Mobile https://sfo15.pathable.com/meetings/302944

Tomorrow is Linaro Networking Group day and the keynotes will both focused on networking.  First up is Dave Neary from Red Hat that will give a talk titled “An NFV Primer: The Next Generation for Telco”  following Dave will be Roger Casals from Symantec Corporation who will give a keynote titled “Security and Trust in a Mobile World”.   To learn more about the agenda tomorrow please visit: https://sfo15.pathable.com/meetings

Day 4 of Linaro Connect SFO15

$
0
0

LNG day image

Linaro Connect San Francisco 2015 day four was another packed day of keynotes and sessions as well as demos.  Thursday was a day dedicated to Linaro Networking Group and included many sessions focused on Open Data Plane, Kernel & Core as well as an afternoon of demos for attendees to view.

Both of the morning keynotes were centered around networking.   First up was  Dave Neary  who works on SDN and NFV community strategy as a member of Red Hat’s Open Source and Standards team and is active in OPNFV.  Dave’s talk was titled “An NFV Primer: The Next Generation for Telco”.  He spoke about changes in the Telco industry and how the industry is more competitive than ever.  He then went on to talk about the promise of NFV.

The second keynote of the day was given by Roger Casals from Symantec, who gave a talk titled “Security and Trust in a Mobile World” touching on both  the theme for the day of networking and the Connect overarching theme of Security.

Following the keynotes the attendees were able to choose from several different tracks of sessions.  Below are some of the sessions that have materials available.

Session ID Session Title Track URL to Session Information
SFO15-400K2: Dave Neary – An NFV Primer: The Next Generation for Telco Keynote https://sfo15.pathable.com/meetings/303069
SFO15-400K3: Roger Casals – Security and Trust in a Mobile World Keynote https://sfo15.pathable.com/meetings/303070
SFO15-401: Mainline on form-factor devices / Improving AOSP  Mobile https://sfo15.pathable.com/meetings/303072
SFO15-402: Architecture & Use of “openCSD” ARM CoreSight Trace decode library  Tools https://sfo15.pathable.com/meetings/303073
SFO15-403: Current state of the LAVA dispatcher Refactoring  Validation https://sfo15.pathable.com/meetings/303074
SFO15-406: ARM FDPIC toolset, kernel & libraries for Cortex-M & Cortex-R mmuless cores Tools https://sfo15.pathable.com/meetings/303078
SFO15-411: Energy Aware Scheduling: Power vs. Performance policy  Power Management https://sfo15.pathable.com/meetings/303083

Be sure to check back in tomorrow for the final day of Linaro Connect.  Tomorrow will start out with a keynote by Karen Sandler on “Ideology in Open Source Compliance” following that will be a Security Panel discussion.    To learn more about the agenda tomorrow please visit: https://sfo15.pathable.com/meetings

Linaro Connect SFO15 week in review

$
0
0
21678923646_7f14ff4c02_k

Linaro Connect celebrated its fifth anniversary at the Linaro Connect San Francisco held September 21-25, 2015.  It was the largest Linaro Connect to date.  There were keynotes each morning as well as many sessions and social events for attendees to select from.  Many of the keynote and session presentations and videos are now available and can be found on the Linaro Connect SFO15 resources page

Announcements

Linaro also made several announcements regarding new members, new groups at Linaro and many product demonstrations.  On Monday George Grey, CEO of Linaro, began his keynote by welcoming the attendees to the fifth anniversary of Linaro Connect and Linaro.  He discussed some of the highlights over the last five years and what to expect going forward.  He then made an announcement about a new group that is being formed by the end of 2015 in Linaro called, Linaro IoT & Embedded Group (LITE) which will focus on  Cortex-A  Linux devices.    There were also several demonstrations around the 96Boards initiative during the keynote.   On Tuesday Linaro announced that two new members have joined Linaro, Acadine Technologies and Tinno Mobile have joined as members of the Linaro Mobile Group (LMG).

Keynotes

Attendees also had the opportunity to attend two keynotes each day by various industry leaders:

Demonstrations

In addition to the keynotes and sessions there was a rich assortment of demonstrations from Linaro and Linaro members, including much of Linaro’s latest ARMv8-A 64-bit software developments.  Demonstrations were given during keynotes as well as on Wednesday during Linaro Enterprise Group day, Thursday during the Linaro Networking Group day and on Friday during the traditional demo Friday held during each Linaro Connect during lunch on the last day.  Below are some of the images from the demonstrations.

Next Linaro Connect

The next Linaro Connect will take place March 7-11, 2016 in Bangkok, Thailand.  To see more about Linaro Connect please visit:  http://connect.linaro.org/bkk16/

BKK16-Banner

Linaro 15.10 Release Available for Download

$
0
0
“Once a word has been allowed to escape, it cannot be recalled.” ~ Horace, Epistles

Linaro 15.10  release is now available for download.  See the detailed highlights of this release to get an overview of what has been accomplished by the Working Groups, Landing Teams and Platform Teams. We encourage everybody to use the 15.10 release.

This post includes links to more information and instructions for using the images. The download links for all images and components are available on our downloads page:

USING THE ANDROID-BASED IMAGES

The Android-based images come in three parts: system, userdata and boot. These need to be combined to form a complete Android install. For an explanation of how to do this please see:

If you are interested in getting the source and building these images yourself please see the following pages:

USING THE UBUNTU-BASED IMAGES

The Ubuntu-based images consist of two parts. The first part is a hardware pack, which can be found under the hwpacks directory and contains hardware specific packages (such as the kernel and bootloader). The second part is the rootfs, which is combined with the hardware pack to create a complete image. For more information on how to create an image please see:

USING THE OPEN EMBEDDED-BASED IMAGES

With the Linaro provided downloads and with ARM’s Fast Models virtual platform, you may boot a virtual ARMv8 system and run 64-bit binaries.  For more information please see:

GETTING INVOLVED

More information on Linaro can be found on our websites:

Also subscribe to the important Linaro mailing lists and join our IRC channels to stay on top of Linaro developments:

KNOWN ISSUES WITH THIS RELEASE

 

  • Bug reports for this release should be filed in Bugzilla (http://bugs.linaro.org) against the individual packages or projects that are affected.

 

Linaro 15.11 Release Available for Download

$
0
0
“Experience is directly proportional to the amount of equipment ruined.” ~ Harrisberger’s Fourth Law of the Lab

Linaro 15.11  release is now available for download.  See the detailed highlights of this release to get an overview of what has been accomplished by the Working Groups, Landing Teams and Platform Teams. We encourage everybody to use the 15.11 release.

This post includes links to more information and instructions for using the images. The download links for all images and components are available on our downloads page:

USING THE ANDROID-BASED IMAGES

The Android-based images come in three parts: system, userdata and boot. These need to be combined to form a complete Android install. For an explanation of how to do this please see:

If you are interested in getting the source and building these images yourself please see the following pages:

USING THE UBUNTU-BASED IMAGES

The Ubuntu-based images consist of two parts. The first part is a hardware pack, which can be found under the hwpacks directory and contains hardware specific packages (such as the kernel and bootloader). The second part is the rootfs, which is combined with the hardware pack to create a complete image. For more information on how to create an image please see:

USING THE OPEN EMBEDDED-BASED IMAGES

With the Linaro provided downloads and with ARM’s Fast Models virtual platform, you may boot a virtual ARMv8 system and run 64-bit binaries.  For more information please see:

GETTING INVOLVED

More information on Linaro can be found on our websites:

Also subscribe to the important Linaro mailing lists and join our IRC channels to stay on top of Linaro developments:

KNOWN ISSUES WITH THIS RELEASE

  • Bug reports for this release should be filed in Bugzilla (http://bugs.linaro.org) against the individual packages or projects that are affected.

 

UPCOMING LINARO CONNECT EVENTS: LINARO CONNECT BANGKOK 2016

Linaro Connect Bangkok 2016 will be held March 7-11, 2016.  More information on this event can be found at: http://connect.linaro.org/bkk16/ 

Linaro Connect Bangkok 2016 Registration Now Open

$
0
0

Mark your calendar for March 7-11, 2016, because it’s the official date of Linaro Connect Bangkok 2016 (BKK16).  It may be 3 months away, but we are already busy planning to make sure this will be the best Linaro Connect yet.  Registration is now open, so claim your spot today.  Register here

Linaro Connect Logo_BKK16 Standard

BKK16 will be held at the 5-star Centara Grand Hotel & Bangkok Convention Centre at CentralWorld. Situated in the very heart of Bangkok’s shopping and business district. The hotel has 55 floors and offers spectacular city views.  A complete lifestyles complex on the 26th floor includes the award-winning SPA Cenvaree along with a fitness centre, tennis courts and an outdoor pool with sundeck.

For shopping and plenty of eateries, the hotel has direct access to CentralWorld, one of the biggest

lifestyle and shopping complexes in Southeast Asia.  You can also access via the Skywalk, the BTS Skytrain, and nearby is the Central Chidlom Department Store and the brand new Central Embassy shopping complex.  The weather in March in Bangkok averages 30°C (85°F) so it will be a great time of year to take a walk out to the shops and restaurants nearby.

Linaro CEO,  George Grey will commence proceedings with a welcome keynote at 8.30am on Monday 7 March. BKK16 is a weeklong event and runs until 2pm Friday 11 March, when we will finish with our Demo Friday technical showcase.  There are so many great things being planned, you can keep up with the latest information about the event on the Linaro Connect website.  The last several Linaro Connects have been sold out weeks prior to the event.  Register now so you do not miss out.


Linaro 15.12 Release Available for Download

$
0
0
“Of course, some people consider hidden bugs to _be_ fixed. I don’t believe in that particular philosophy myself.” ~ Linus Torvalds

Linaro 15.12  release is now available for download.  See the detailed highlights of this release to get an overview of what has been accomplished by the Working Groups, Landing Teams and Platform Teams. We encourage everybody to use the 15.12 release.

This post includes links to more information and instructions for using the images. The download links for all images and components are available on our downloads page:

USING THE ANDROID-BASED IMAGES

The Android-based images come in three parts: system, userdata and boot. These need to be combined to form a complete Android install. For an explanation of how to do this please see:

If you are interested in getting the source and building these images yourself please see the following pages:

USING THE UBUNTU-BASED IMAGES

The Ubuntu-based images consist of two parts. The first part is a hardware pack, which can be found under the hwpacks directory and contains hardware specific packages (such as the kernel and bootloader). The second part is the rootfs, which is combined with the hardware pack to create a complete image. For more information on how to create an image please see:

USING THE OPEN EMBEDDED-BASED IMAGES

With the Linaro provided downloads and with ARM’s Fast Models virtual platform, you may boot a virtual ARMv8 system and run 64-bit binaries.  For more information please see:

GETTING INVOLVED

More information on Linaro can be found on our websites:

Also subscribe to the important Linaro mailing lists and join our IRC channels to stay on top of Linaro developments:

KNOWN ISSUES WITH THIS RELEASE

  • Bug reports for this release should be filed in Bugzilla (http://bugs.linaro.org) against the individual packages or projects that are affected.

 

UPCOMING LINARO CONNECT EVENTS: LINARO CONNECT BANGKOK 2016

Linaro Connect Bangkok 2016 will be held March 7-11, 2016.  More information on this event can be found at: http://connect.linaro.org/bkk16/ 

LHG takes another step forward in Enhanced Content Protection with OPTEE on ARM® TrustZone®

$
0
0

LHG takes another step forward in Enhanced Content Protection with OPTEE on ARM® TrustZone®

Mark Gregotski and Zoltan Kuscsik

As studios begin creating ultra-high definition (UHD) content and delivering it to customer devices, a higher level of content protection is required. As a consequence of these more stringent requirements, the digital rights management (DRM) solutions that protect the content, as well as the keys used to decrypt the content, are moving from software to hardware based solutions. Hardware-based security systems allow the security and robustness rules for premium content to be satisfied.

A secure Trusted Execution Environment (TEE) allows the security processes related to key management, content decryption, and content decoding to be executed in a secure environment, not accessible from user space. An example implementation of a TEE, is the Linaro OPTEE that runs on ARM-based CPU TrustZone® architecture.

In addition, commercial DRM porting kits are now available that interface to a TEE to take advantage of this secure hardware functionality. An example, is the latest Microsoft PlayReady® DRM porting kit designed to interface to a generic TEE. The PlayReady interface for Trusted Execution Environments (PRiTEE) is implemented in the PlayReady PK 3.0 release.

PRiTEE Slides

 

PlayReady Device Based Content Protection Improvements [1]

In September of 2015, the Linaro Digital Home Group (LHG) provided the first integration of PlayReady PK3.0 with OPTEE [2]. In that implementation, the PlayReady PK interfaced with OPTEE and the content decryption was performed by a Trusted Application (TA) running in OPTEE using the OPTEE decryption functions.

In the recent implementation, LHG has completed the entire integration of the PlayReady DRM libraries with OPTEE. LHG created a GlobalPlatform compliant TA running on OPTEE that can fully encapsulate the PlayReady DRM as a static library.

The GlobalPlatform (GP) based trusted applications have multiple parts:
API for the normal (non-secure) world. This is the GP TEE Client API (v1.0).
API for the secure world. This is the GP TEE Internal Core API (v1.1). See [3] for both API specifications.

For the non-secure world we pass the PlayReady data using GP to the TA, and we are using various GP secure functions to implement the OEM specific specific requirements for the PlayReady TA (such as, memory allocation using GP memory allocation functionality).

In this solution, the license parsing, key management and the content decryption is performed inside the secure context. This architecture ensures that the TA is aligned with the PlayReady architecture requirements for hardware level DRM protection.

The Open CDMI [4] is an open interface that enables the integration of the DRM/CDM into a browser or media application. The OpenCDM module communicates with the TA using the Microsoft PRiTEE interface and exposes the Open CDMI interface for decrypting PlayReady protected content using the Encrypted Media Extensions in Chromium.

PRiTEE Slides 2
High level overview of Media and Security Components

There is a demo running on the STMicroelectronics B2120 reference platform and we are in the process of porting this build to the HiKey 96Boards platform.

The latest solution is accessible now to our members who are PlayReady Licensees; so we look forward to see it running on multiple ARM-based hardware platforms in the upcoming months on both Android and Linux.

The next stage that LHG is working on is the implementation of a secure video path that uses a secure buffer memory allocation framework in TrustZone for the media pipeline. Stay tuned for the latest updates in the LHG security solution development.

  1. https://www.microsoft.com/playready/features/EnhancedContentProtection.aspx
  2. http://www.linaro.org/news/linaro-and-microsoft-collaborate-on-secure-media-solutions-for-arm-based-socs/
  3. http://globalplatform.org/specificationsdevice.asp
  4. https://github.com/kuscsik/linaro-cdmi

Linaro 16.01 Release Available for Download

$
0
0
“Digital circuits are made from analog parts.” ~ Don Vonada

Linaro 16.01  release is now available for download.  See the detailed highlights of this release to get an overview of what has been accomplished by the Working Groups, Landing Teams and Platform Teams. We encourage everybody to use the 16.01 release.  To sign-up for the release mailing list go here:  https://lists.linaro.org/mailman/listinfo/linaro-release 

This post includes links to more information and instructions for using the images. The download links for all images and components are available on our downloads page:

USING THE ANDROID-BASED IMAGES

The Android-based images come in three parts: system, userdata and boot. These need to be combined to form a complete Android install. For an explanation of how to do this please see:

If you are interested in getting the source and building these images yourself please see the following pages:

USING THE UBUNTU-BASED IMAGES

The Ubuntu-based images consist of two parts. The first part is a hardware pack, which can be found under the hwpacks directory and contains hardware specific packages (such as the kernel and bootloader). The second part is the rootfs, which is combined with the hardware pack to create a complete image. For more information on how to create an image please see:

USING THE OPEN EMBEDDED-BASED IMAGES

With the Linaro provided downloads and with ARM’s Fast Models virtual platform, you may boot a virtual ARMv8 system and run 64-bit binaries.  For more information please see:

GETTING INVOLVED

More information on Linaro can be found on our websites:

Also subscribe to the important Linaro mailing lists and join our IRC channels to stay on top of Linaro developments:

KNOWN ISSUES WITH THIS RELEASE

  • Bug reports for this release should be filed in Bugzilla (http://bugs.linaro.org) against the individual packages or projects that are affected.

 

UPCOMING LINARO CONNECT EVENTS: LINARO CONNECT BANGKOK 2016

Linaro Connect Bangkok 2016 will be held March 7-11, 2016.  More information on this event can be found at: http://connect.linaro.org/bkk16/ 

Testing a Trusted Execution Environment

$
0
0

core-dump

Why you need to test your software is quite obvious and therefore this blog post will not be about that, instead I’ll talk a little bit about how we are testing OP-TEE using a tool called xtest (optee_test). I will also talk about what components are involved, what kind of tests are performed, what is missing, etc. But first let’s start with a short background.

Background

Just as the other components in OP-TEE the test framework also has origins from ST-Ericsson and STMicroelectronics. A couple of years ago, when OP-TEE was being developed, the developers were engaged in GlobalPlatform testing, in the so called TestFest (for simplicity let’s call it OP-TEE even though it strictly isn’t correct, since back then the TEE solution didn’t really have a name, it was the ST-Ericsson TEE solution). At this time there were no official test suite nor compliance program ready and the goals with the TestFests were twofold, first to ensure that the different TEE vendors respective TEE solution was behaving according to the specification, secondly that the test tool(s) and the specifications themselves were correct. At the same time as this work took place there were quite a few “standalone” test cases being implemented as a complement to the GlobalPlatform tests. The nature of those were more to address the missing pieces in GlobalPlatform and to test corner cases, hardware- and extended features. So side by side the engineers at ST-Ericsson were running their own tests as well as the tests provided by the ones in charge of GlobalPlatform compliance program.

figure1

Figure 1: Output from xtest

Linaro gets into the picture

When Linaro got involved in the development of OP-TEE we also had a need for testing the code we are developing and at the same time we would like to give our members the ability to use a suitable test framework. The only problem was that the test cases coming from GlobalPlatform couldn’t be shared with anyone (including Linaro) since to get access to those, a company either had to be a member of GlobalPlatform or it had to purchase the needed files directly from GlobalPlatform. Therefore the engineers from ST immediately started working on separating  the tests implemented by themselves from the ones that they had gotten from GlobalPlatform. When that job was completed, they shared their own developed tests with Linaro engineers and the members of Linaro. This piece of test code is what we today refer to when saying “the standard test” and that is also what you can find on GitHub since a couple of months ago on the OP-TEE project in the git called optee_test. That repository is no longer private to Linaro and its members.

 

For OP-TEE development we have configured our repositories at GitHub so that a pull request will trigger a Travis job which in turn would automatically trigger builds for all supported platforms. In addition to that we always will automatically run xtest using QEMU (here is an example of how that could look like). In the long run we would like to also start using our own Linaro infrastructure (Jenkins + LAVA) as a complement to Travis so that we could do automatic testing on all the devices we are supporting in OP-TEE.

Software components

The test framework consists of a host application, which is a normal user space application running in Linux. This is the piece of software that initiates and runs the actual tests and gathers test results etc. When it comes to Linux kernel there are no changes at all. It’s still the same TEE driver in use that is responsible for transporting the data back and forth between normal world, user space and secure world. Likewise on the secure side, there are no changes to the secure OS itself (TEE core). Instead all the code specific totesting will be performed as a set of different Trusted Applications (I’ll go more into details further down in this blog post).

 

Host application

The host application, which by the way is the one we call “xtest”, has been divided into a couple of different files where each file corresponds to a certain area or feature. As of writing this, you will find the following files for the host application (there are a few more files, but those other files are the application and test framework itself):

  • xtest_1000.c: contains the OS related tests basic OS features, panics, wait functionality, RPC messaging, signature header verification tests by loading a fake and a corrupt Trusted Application. It also tests invalid memory access and concurrent usage of Trusted Applications.
  • xtest_4000.c: contains all crypto related testing. It is basically testing crypto APIs that are exposed to the Trusted Application via the GlobalPlatform Internal TEE core specification.
  • xtest_5000.c: this file have tests for shared memory handling.
  • xtest_6000.c: contains test for storage, which exercises the GlobalPlatform secure storage API as well as the underlying “POSIX” file system API.
  • xtest_7000.c: This also contains tests for shared memory etc. However, this is something that will only be used when having access to and enabling the tests coming from GlobalPlatform (more on that further down in this blog post).
  • xtest_10000.c has test code containing extensions going beyond the GlobalPlatform specifications. For example, this is where we are testing key derivation functionality like PBKDF2, HKDF and Concat KDF.
  • xtest_20000.c this file also has tests related to storage, but this time those are more aimed at the secure storage implementation as such and they verify that files are actually being written to the file system, checking that they haven’t been corrupted and that they are being deleted etc. As an example, when initiating a store operation from secure world there should be file(s) created in Linux and accessible at /data/tee/{directory}/{filename}/block.xxx.
  • xtest_benchmark_1000.c: This is so far the only file related to benchmarking and it contains a couple of benchmark tests for the secure storage implementation.

 

The main function could be found in the file xtest_main.c. This file basically just lists all test cases that should be enabled, and parses a few command line arguments followed by starting the actual tests. If you dive into the test code itself, you will see that the test framework itself implements macros that are used to evaluate if the test has passed or failed. You will, for example, frequently see ADBG_EXPECT_TEEC_SUCCESS, ADBG_EXPECT, ADBG_EXPECT_TEEC_ERROR_ORIGIN, ADBG_EXPECT_TRUE and ADBG_EXPECT_TEEC_RESULT everywhere in the test code. There are others, but those are the most commonly used.

 

On a top level, a test case is added using the macro ADBG_CASE_DEFINE and that is what you can see on the top in each and every file listed above. As arguments, this macro takes a test label, a function pointer, a title, a short description of what it is testing, requirement ID and a short description of how it will be tested. As an example, have a look at XTEST_TEE_10001 which is defined here. As you can see, this particular test is supposed to test functionality related to key derivation.

 

Within each test you can define sub-tests and to do so you have to wrap your code in-between Do_ADBG_BeginSubCase() and Do_ADBG_EndSubCase() calls. This isn’t something you strictly need to do, but it is a nice way of splitting up the tests into manageable sections, that will help better pinpointing where something went wrong in case of a test case failure. The number of tests and subtests is also something that will be presented when all test cases have been run (see Figure 3 further down).

Test Trusted Applications

As I’ve mentioned above, all code related to testing could be found within a set of Trusted Applications. Below is a list of the Trusted Applications that are used by xtest.

 

  • concurrent: The concurrent Trusted Application is responsible for testing the ability to run several Trusted Applications simultaneously – a feature that has been merged into OP-TEE quite recently. For the host application you will find this application’s code in the xtest_1000.c file.
  • create_fail_test: This is a tiny little TA used solely to test OP-TEE’s behaviour when loading a corrupt or fake Trusted Application.
  • crypt: Despite the fact that there is the crypto API defined by GlobalPlatform, in OP-TEE, this particular Trusted Application also contains an AES-ECB and a SHA-256 (224) implementation within the TA itself, that is mostly due to historic reasons. But the majority of the entry points are calling GlobalPlatform Internal API functions. This Trusted Application tests MAC, AAED, hashes, ciphers, random number generator etc.
  • os_test: Mainly tests OS related features such as memory access rights, properties, time API and floating point operations as well as the MPA library (implementing big numbers).
  • rpc_test: Test that the RPC mechanism and loading of other Trusted Applications are working properly. It does this by letting the TA itself calling functionality in the crypt TA which will trigger loading of the crypt TA using RPC messages.
  • sims: Testing the Single Instance and Multiple Session features specified by GlobalPlatform.
  • storage: Contains tests related to the (secure) storage functionality. It tests all the functions of the GlobalPlatform specification that cover the so called “Persistent Objects”. On a high level or in Unix terms, this can be seen as the POSIX API (in reality there is a POSIX level behind the GP interfaces).
  • storage_benchmark: As the name indicates, this TA benchmarks storage operations. It reads and writes data of various chunk sizes and then in the end creates a performance report.

 

What about the tests coming from GlobalPlatform?

The compliance test suite (GlobalPlatform TEE Initial Configuration Compliance Test Suite v1.1.0.4) that can be purchased from GlobalPlatform (free for GP members) consists of a compliance adaptation layer specification that needs to be implemented to run the tests. It also contains a set of configuration files, more specifically – XML files specifying how functions should be called, what parameters to pass to them and what kind of test results to expect, i.e., you will not get any actual code that is ready to be compiled. How those XML files will end up being used is up to the end user. What we did early on was to configure xtest, so that it would be easy to extend it later to also include the compliance test suite from GlobalPlatform. So by putting the XML files on a certain path, using the adaptation layer, installing a couple of tools (xalan) and running make with the “patch” as an argument, there will be a set of new Trusted Applications as well as patch xtest itself to also include the compliance tests. I.e., the XML files will be transformed into C code in this step. After performing that step you will not only run the so called standard test, but you will also run the compliance tests from GP in the same run.

figure2

Figure 2: xtest overview

Licenses

One has to be careful when working with xtest, since there are different licenses in use in different areas. In general we usually use BSD Clause-2 license for most of our code. But in this case, when it comes to test related code, we’re using both BSD Clause-2 and GPLv2 license. All code running on secure side in the standard tests (Trusted Applications) are using the BSD Clause-2 license while the code running in normal world is using GPLv2 license. The same is true for the code used when extending xtest, however we must also follow the license stated by GlobalPlatform (GlobalPlatform Compliance License Agreement). In figure 2 below, you can see more clearly how xtest is divided and what licenses are in use.

Shortcomings and future improvements

Today xtest is a test framework that does API testing of the exposed functionality for the Client API and for the Internal Core API. It contains quite a few test cases. Running the standard test on QEMU (Intel Core i5-4670K CPU @ 3.40GHz) results in the following:

figure3

Figure 3: xtest standard test result

If you also enable the GP compliance tests, then you get even better coverage. So the APIs as such are being thoroughly tested and that is all good. However! Since it is security we’re dealing with here, we still have a lot to do when it comes to performing a focused security testing. There exist both concepts and tools and even companies solely dedicated to white box testing, where the goal is to find bugs and potential vulnerabilities in the code. For example, over the years people have found numerous bugs in Linux kernel by using Trinity (fuzz tester). With Trinity the main goal isn’t strictly about enhancing security but rather to ensure that the system calls in Linux kernel are robust. A crash (Linux kernel oops) can in some cases also be an entrance point for a kernel exploit and therefore it is still important to find and fix issues discovered by such tools as Trinity. Having something similar running on the secure side would probably be really useful. We have heard that GlobalPlatform will include fuzz testing in a new test suite that is currently being developed (draft is available for GP members here TEE Security Test Suite v0.1.0).

 

There are also side channel attacks. Some side channel attacks, like power analysis, cannot be done in software only, but still it would be worth adding tests covering such cases when possible. For example, timing attacks are something one can do using only software and having test cases automatically performing timing attacks would be very useful. Since we mainly use ARM TrustZone™ it would also be worth adding tests covering the boundaries between the two worlds. I.e add tests that ensure that memory is or isn’t accessible from the other side. There are some memory region tests in xtest already today, but it would be great to  add more tests in this area. With some imagination one could also start to play with TrustZone Address Space Controller and add tests that ensure that the configuration of that system IP behaves as expected.

Final words

I hope this post gave a useful  introduction to xtest and explained how we are testing OP-TEE. xtest sources  is also a good source to look at if you want to know more about how to write Trusted Applications and how to use the GlobalPlatform APIs. We are continuously adding tests and hopefully sooner than later we will also address the shortcomings mentioned above. But since most of it is open source and thereby freely available, we would be more than happy seeing people with experience in this area getting involved by giving feedback, coming up with ideas and maybe even submitting patches that improve xtest.

Linaro Connect Bangkok 2016 – Scheduling…

$
0
0

Going to BKK16? You will need to know Pathable

Linaro Connect BKK16 is just around the corner and we are expecting another great event.  There will be daily keynotes given by several industry leaders, numerous interesting sessions that attendees will want to participate in, afternoon hacking and a lot of meetings.  So how do you keep track of everything happening each day?  You will need to logon to the very easy to use Pathable event schedule system.  

Pathable allows attendees to see the daily schedule of all activities, select just the ones they would like to attend, schedule meetings, have discussions with other attendees and receive important notices of any changes or additions to the schedule.  It is also where all the content for each session will be posted first, so any links to presentations or videos will be posted under each session as soon as they are available.  

This site will be the place to help you get the most out of your time at Linaro Connect BKK16 so it is really important that you:

  1. Go to the site and login prior to the event
  2. Set-up your profile and become familiar with the site and how it works
  3. There will be a “mobile app” available for you to install from the app store to access the community.  This will be coming shortly.  We will let attendees know as soon as this is available.

Below are some steps to help get you started:

Step 1:  Go to the pathable site:  https://bkk16.pathable.com/

Step 2:  Login – If you have registered for Linaro Connect BKK16  you should have received an email from pathable after 12 February with your login details. This email was entitled: “Please join Linaro Connect Bangkok 2016″. If you cannot find this email please send an email to: connect@linaro.org

Step 3:  Once you get logged in – You will want to update your profile page so that other attendees know who you are, as well as set-up your password to something you can remember easily.  

Step 4:  Add Tags to your profile to identify teams you work with or areas you are interested in.  You can see a list of the current tags being used by going to:  Attendees>Tags (in the left hand drop down), but you can add your own as well.  When people perform a “search” in the attendees or members list, they will be able to choose from the networking interests and tags you and the rest of the community have added to their profiles.

Step 5:  Review the current schedule https://bkk16.pathable.com/meetings  and add sessions you would like to attend to your schedule https://bkk16.pathable.com/meetings/my.  

  • Click the Schedule tab
  • Hover your mouse over the session description box that pops up that you would like to select and click the “Add” button that appears.
  • If you are on a tablet computer or wish to read more before making your choice, select the name of the session you are interested in. Then click the “Add to My Agenda” button to add the session to your schedule.
  • You can then go to the My Schedule page to see your agenda for each day

Step 6:  If you need to have any meetings with other attendees you can also easily set those up through Pathable.  

  • Visit the profile page of the person you would like to meet
  • Click the Schedule a Meeting option
  • If you would like to add additional people to the meeting, type their names into the “Attendees” field
  • You will see the schedule and free-busy time for you and the invitees on the right. Use your mouse to select a time that is mutually available.
  • Select or enter a Location for your meeting*
  • Type “Subject” and “Description” for the meeting to give the invitees background on why you would like to meet
  • Click the Send Invitation button.

​Your recipients will receive an email notification for the meeting with the option to accept or decline your request. You will receive an email notification when they have responded.

*Once meeting rooms are booked, there are still many areas around the hotel to meet.  There is the option to have your attendees meet at the registration desk or put in your own location.  If you need a specific room please see the registration desk.

For more help with Pathable:

These are the basics for Pathable however, there is much more you can do with the tool; including messaging other attendees, having a group chat, adding documents for a sessions etc.  To get more details you can visit the Pathable Knowledge Base to learn more.  http://support.pathable.com/knowledgebase

 

Viewing all 179 articles
Browse latest View live