01403 784500 sales@harmonicss.co.uk

Asymmetric Multi Processing (AMP) using VxWorks 6.9

Asymmetric Multi Processing (AMP) allows a multiprocessor system to run multiple Operating Systems (OS) that are independent of each other. In other words, each CPU has it’s own private memory space, which contains the OS and the applications that are to run on that CPU. In addition, there can be some shared memory space that is used for multiprocessor communication. This is contrasted with Symmetric Multiprocessing (SMP), in which one OS runs on multiple CPUs using a public shared memory space.

This tutorial is the product of recent work on the Freescale P4080 Processor. If you are not familiar with this chip, it is an 8 core beast, in the PowerPC family. (The manual is over 3500 pages!)

The figure below shows a typical AMP system. Once CPU0 has booted up, the wrload utility is used to load a specific vxWorks image from the filesystem into the memory space of the target CPU. CPU 0 has a larger portion of the available memory space, whereas the other CPUs have a smaller region to play with.

VxWorks AMP System showing private and shared memory allocation

VxWorks AMP System showing private and shared memory allocation

There are also two different flavours of AMP, supervised (hypervisor) and unsupervised.  This article will deal only with unsupervised AMP. Hypervisor is a whole different kettle-o-fish Wink

Why Use AMP? 

AMP lends itself to applications that can be split into logical modules (such as control plane and data plane processing in a router), or perhaps where an increased level of security is needed on some processors (devices are assigned to processors, so the IO connections and secure applications can be seperated by the MMU). In addition, completely different OSs can be loaded onto the CPUs, so you might want a mix of linux and different builds of vxWorks.

AMP also helps when upgrading old systems by consolidating multiple boards onto a single board. The system design can remain the same, and porting becomes much easier, by taking an old application that ran on an old board, and porting it to run on on one of the CPUs in a multiprocessor system. By using AMP, there is no need to merge all the application code into a single code space. Existing Ethernet connections can even be preserved with no changes by using MIPC Network Devices (MND).

Building AMP in VxWorks 

You have two options for building VxWorks images for AMP, namely using WindRiver Workbench (the GUI), or using the command line. The trouble with using Workbench is that for 8 CPUs, you have to configure eight VxWorks images individually, and this gets very tedious. Instead, create a makefile like this one, in /workspace:

############################################################################### 
#  
# makefile for amc718 AMP builds 
# 
# Ed Liversidge, Harmonic Software Systems Ltd 
# 
# http://www.harmonicss.co.uk  
#  
# This makefile demonstrates how to build an AMP system for the  
# Vadatech AMC718 board. This board uses an 8 core Freescale P4080 Chip 
# 
# From a vxWorks development shell, type make amp. VxWorks images will be copied 
# into FTP_DIR 
# 

# for Windows hosts, fix slashes
WIND_HOME := $(subst \,/,$(WIND_HOME))
WIND_BASE := $(subst \,/,$(WIND_BASE))
WIND_USR := $(subst \,/,$(WIND_USR))

TOOL     = gnu

# all vxWorks images will get copied here for pickup
FTP_DIR  = C:/temp/incoming

AMP0_DIR  = amc718_gnu_amp0
AMP0_PROJ = $(AMP0_DIR)/$(AMP0_DIR).wpj
AMP1_DIR  = amc718_gnu_amp1
AMP1_PROJ = $(AMP1_DIR)/$(AMP1_DIR).wpj
AMP2_DIR  = amc718_gnu_amp2
AMP2_PROJ = $(AMP2_DIR)/$(AMP2_DIR).wpj
AMP3_DIR  = amc718_gnu_amp3
AMP3_PROJ = $(AMP3_DIR)/$(AMP3_DIR).wpj
AMP4_DIR  = amc718_gnu_amp4
AMP4_PROJ = $(AMP4_DIR)/$(AMP4_DIR).wpj
AMP5_DIR  = amc718_gnu_amp5
AMP5_PROJ = $(AMP5_DIR)/$(AMP5_DIR).wpj
AMP6_DIR  = amc718_gnu_amp6
AMP6_PROJ = $(AMP6_DIR)/$(AMP6_DIR).wpj
AMP7_DIR  = amc718_gnu_amp7
AMP7_PROJ = $(AMP7_DIR)/$(AMP7_DIR).wpj

amp: amp0 amp1 amp2 amp3 amp4 amp5 amp6 amp7

amp0:
vxprj create -force $(BSP) $(TOOL) $(AMP0_PROJ)
vxprj bundle add $(AMP0_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add $(AMP0_PROJ) BUNDLE_AMP_PRI
vxprj component add $(BSP_PROJ) INCLUDE_PCI_BUS
vxprj build $(AMP0_PROJ)
cp $(AMP0_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.0

amp1:
vxprj create -force $(BSP) $(TOOL) $(AMP1_PROJ)
vxprj bundle add  $(AMP1_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add  $(AMP1_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP1_PROJ) INCLUDE_AMP_CPU_01
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP1_PROJ) INCLUDE_WDB_SYS
vxprj build $(AMP1_PROJ)
cp $(AMP1_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.1

amp2:
vxprj create -force $(BSP) $(TOOL) $(AMP2_PROJ)
vxprj bundle add  $(AMP2_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add  $(AMP2_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP2_PROJ) INCLUDE_AMP_CPU_02
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP2_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices
vxprj component remove $(AMP2_PROJ) INCLUDE_NETWORK
vxprj build $(AMP2_PROJ)
cp $(AMP2_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.2

amp3:
vxprj create -force $(BSP) $(TOOL) $(AMP3_PROJ)
vxprj bundle add  $(AMP3_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add  $(AMP3_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP3_PROJ) INCLUDE_AMP_CPU_03
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP3_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices
vxprj component remove $(AMP3_PROJ) INCLUDE_NETWORK
vxprj build $(AMP3_PROJ)
cp $(AMP3_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.3

amp4:
vxprj create -force $(BSP) $(TOOL) $(AMP4_PROJ)
vxprj bundle add  $(AMP4_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add  $(AMP4_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP4_PROJ) INCLUDE_AMP_CPU_04
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP4_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices.
vxprj component remove $(AMP4_PROJ) INCLUDE_NETWORK
vxprj build $(AMP4_PROJ)
cp $(AMP4_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.4

amp5:
vxprj create -force $(BSP) $(TOOL) $(AMP5_PROJ)
vxprj bundle add  $(AMP5_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add  $(AMP5_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP5_PROJ) INCLUDE_AMP_CPU_05
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP5_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices.
vxprj component remove $(AMP5_PROJ) INCLUDE_NETWORK
vxprj build $(AMP5_PROJ)
cp $(AMP5_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.5

amp6:
vxprj create -force $(BSP) $(TOOL) $(AMP6_PROJ)
vxprj bundle add  $(AMP6_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add  $(AMP6_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP6_PROJ) INCLUDE_AMP_CPU_06
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP6_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices.
vxprj component remove $(AMP6_PROJ) INCLUDE_NETWORK
vxprj build $(AMP6_PROJ)
cp $(AMP6_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.6

amp7:
vxprj create -force $(BSP) $(TOOL) $(AMP7_PROJ)
vxprj bundle add  $(AMP7_PROJ) BUNDLE_STANDALONE_SHELL
vxprj bundle add  $(AMP7_PROJ) BUNDLE_AMP_SEC
vxprj component add $(AMP7_PROJ) INCLUDE_AMP_CPU_07
# need to remove INCLUDE_WDB_SYS, incompatible with AMP
vxprj component remove $(AMP7_PROJ) INCLUDE_WDB_SYS
# remove networking. No devices.
vxprj component remove $(AMP7_PROJ) INCLUDE_NETWORK
vxprj build $(AMP7_PROJ)
cp $(AMP7_DIR)/default/vxWorks $(FTP_DIR)/vxWorks.7

Using wrload

The wrload utility is executed on the vxWorks shell on the first cpu (cpu0). It will load a vxWorks image from a mounted filesystem, load the code and data onto the target cpu, and then zero the .bss segment. I have found that it can be sensitive to consecutive loads, so I tend to put taskDelays between calls. Also, the vxWorks expects some sort of bootline, and doesn’t respond well when one is missing. See the example below:

wrload ("-file host:/vxWorks.1 -cpu 1 -tsym \"*sysBootLine=dtsec(1,1) host:/vxWorks.1 h=192.168.1.100  e=192.168.1.51\" ");
taskDelay (60);
wrload ("-file host:/vxWorks.2 -cpu 2 -tsym \"*sysBootLine=dtsec(2,2) \" ");
taskDelay (60);

Using ‘tip’ to get a Shell on a CPU

tip is a new function that connects to the other CPUs via a MSD (MIPC Serial Device), enabling a shell to be established where there isn’t a physical serial connection available. The MSD connections are automatically created by the BSP:

-> devs
drv name
  0 /null
  1 /tyCo/0
  1 /tyCo/1
  1 /ttyMsd0
  1 /ttyMsd1
  1 /ttyMsd2
  1 /ttyMsd3
  1 /ttyMsd4
  1 /ttyMsd5
  1 /ttyMsd6

To connect to CPU1 and CPU2 (the first msd connection, msd0, will connect to cpu1):

tip ("dev=/ttyMsd0", "dev=/ttyMsd1")
~?
[Available commands]
~.: Drop connection and exit
~l: List available sessions
~s: Switch session
~?: Print command summary

Conclusion 

Hopefully this has given you a flavour of using AMP under VxWorks 6.9. Of course we are available to help you with your Multiprocessor BSPs, MIPC/MSD setup and configuration, or wrestling with the beast that is the Freescale P4080. Just contact us at sales@harmonicss.co.uk or get a quick quote

This page has been translated into Serbo-Croatian : http://science.webhostinggeeks.com/asimetricni-multi-processing-amp

This page has also been translated into Blugarian : http://go-grafix.com/blog/asimetrichna-multi-obrabotka-amp-izpolzvashhi-vxworks-6-9/