SLARToolkit 1-2-3 logo

Beginner’s Guide

This step by step guide explains how to build a basic Silverlight SLARToolkit application with the CaptureSourceMarkerDetector. The Marker Detectors documentation shows how to use the other Marker Detector implementations.
Microsoft Visual Studio 2010 (Express is sufficient), the Silverlight 4 Tools for Visual Studio and a standard webcam are required.
The final Visual Studio solution of this guide's application is available in the Source Code repository: trunk\SLARToolKit\Solution\SLARToolKitBeginnersGuide.sln


1. Start Visual Studio and create a new Silverlight Application.

New Project

2. Select at least Silverlight 4 as the Silverlight Version and uncheck the Host the Silverlight application in a new web site CheckBox.

New Silverlight application

3. Download the latest SLARToolkit binaries and add a reference to the SLARTooklit.dll and Matrix3DEx.dll.

Add Reference

4. Open the MainPage.xaml file and add an event handler to the Loaded event (UserControl_Loaded).

<UserControl x:Class="SLARToolKitBeginnersGuide.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="550" d:DesignWidth="700"

    Loaded="UserControl_Loaded">

Add a StackPanel to the LayoutRoot Grid and a TextBlock, a Grid with an Rectangle and a Button as children of the StackPanel. Also add an event handler to the Button's Click event (Button_Click).
The video stream from the webcam will be shown inside the Rectangle with the help of a VideoBrush. The Button is used to ask the user for the webcam access and to start the capturing.

<!-- ... -->
    <Grid x:Name="LayoutRoot" Background="White">
        <StackPanel>
            <TextBlock Text="SLARToolkit - Silverlight Augmented Reality Beginner's Guide" HorizontalAlignment="Center" />
            <Grid Width="640" Height="480">
                <Rectangle Name="Viewport" Stroke="Black" StrokeThickness="2" />
            </Grid>
            <Button Content="Start Fun" HorizontalAlignment="Center" Click="Button_Click" />
        </StackPanel>
    </Grid>

The complete XAML file should now look like this:

<UserControl x:Class="SLARToolKitBeginnersGuide.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="550" d:DesignWidth="700"
    Loaded="UserControl_Loaded">

    <Grid x:Name="LayoutRoot" Background="White">
        <StackPanel>
            <TextBlock Text="SLARToolkit - Silverlight Augmented Reality Beginner's Guide" HorizontalAlignment="Center" />
            <Grid Width="640" Height="480">
                <Rectangle Name="Viewport" Stroke="Black" StrokeThickness="2" />
            </Grid>
            <Button Content="Start Fun" HorizontalAlignment="Center" Click="Button_Click" />
        </StackPanel>
    </Grid>
</UserControl>


5. Open the code-behind (MainPage.xaml.cs), add a member variable of the type CaptureSource, then navigate to the Loaded event handler (UserControl_Loaded), initialize the CaptureSource with the default video device and fill the Rectangle with a VideoBrush that has its source set to the CaptureSource.

CaptureSource captureSource;

private void UserControl_Loaded(object sender, RoutedEventArgs e)
{
   // Initialize the webcam
   captureSource = new CaptureSource();
   captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();

   // Fill the Viewport Rectangle with the VideoBrush
   var vidBrush = new VideoBrush();
   vidBrush.SetSource(captureSource);
   Viewport.Fill = vidBrush;
}

Navigate to the Button_Click event handler, request the webcam device access and start the capturing.

private void Button_Click(object sender, RoutedEventArgs e)
{
   // Request webcam access and start the capturing
   if (CaptureDeviceConfiguration.RequestDeviceAccess())
   {
      captureSource.Start();
   }
} 

The complete code-behind should now look like this:

using System.Windows;
using System.Windows.Controls;
using System.Windows.Media;

namespace SLARToolKitBeginnersGuide
{
   public partial class MainPage : UserControl
   {
      CaptureSource captureSource;

      public MainPage()
      {
         InitializeComponent();
      }

      private void UserControl_Loaded(object sender, RoutedEventArgs e)
      {
         // Initialize the webcam
         captureSource = new CaptureSource();
         captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();

         // Fill the Viewport Rectangle with the VideoBrush
         var vidBrush = new VideoBrush();
         vidBrush.SetSource(captureSource);
         Viewport.Fill = vidBrush;
      }

      private void Button_Click(object sender, RoutedEventArgs e)
      {
         // Request webcam access and start the capturing
         if (CaptureDeviceConfiguration.RequestDeviceAccess())
         {
            captureSource.Start();
         }
      }
   }
}


6. Start Debugging (F5 key). The Silverlight application should now load inside your default browser.
If you click the Start Fun Button for the first time you need to give your permission for the capturing. This application uses the default Silverlight capture device. You can specify the video and audio devices that are used by default with the Silverlight Configuration. Just press the right mouse button over the application, click Silverlight in the context menu and select the Webcam / Mic tab to set them.
You should now see the stream from the webcam inside the Rectangle.


7. In the MainPage.xaml file, add a Canvas after the Rectangle, and add a TextBox inside the Canvas.

<!-- ... -->
            <Grid Width="640" Height="480">
                <Rectangle Name="Viewport" Stroke="Black" StrokeThickness="2" />
                <Canvas>
                    <TextBox Name="Txt" Text="Silverlight 4 rockz!" Width="300" Height="300" TextWrapping="Wrap" AcceptsReturn="True" FontSize="16" />
                </Canvas>
            </Grid> 
<!-- ... -->

The final XAML:

<UserControl x:Class="SLARToolKitBeginnersGuide.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    d:DesignHeight="550" d:DesignWidth="700"
    Loaded="UserControl_Loaded">

    <Grid x:Name="LayoutRoot" Background="White">
        <StackPanel>
            <TextBlock Text="SLARToolkit - Silverlight Augmented Reality Beginner's Guide" HorizontalAlignment="Center" />
            <Grid Width="640" Height="480">
                <Rectangle Name="Viewport" Stroke="Black" StrokeThickness="2" />
                <Canvas>
                    <TextBox Name="Txt" Text="Silverlight 4 rockz!" Width="300" Height="300" TextWrapping="Wrap" AcceptsReturn="True" FontSize="16" />
                </Canvas>
            </Grid>
            <Button Content="Start Fun" HorizontalAlignment="Center" Click="Button_Click" />
        </StackPanel>
    </Grid>
</UserControl>


8. Download the SLAR marker pattern file and add it to the project (Shift+Alt+A). Set the file's Build Action property to Resource.

Include pattern file

9. Include the SLARToolkit and System.Windows.Media.Media3D namespaces in the MainPage.xaml.cs code-behind file. Also add a member variable of the type CaptureSourceMarkerDetector and append the following code to the Loaded event handler:

// ...

using SLARToolKit;
using System.Windows.Media.Media3D;

namespace SLARToolKitBeginnersGuide
{
   public partial class MainPage : UserControl
   {      
      // ...

      CaptureSourceMarkerDetector arDetector;

      private void UserControl_Loaded(object sender, RoutedEventArgs e)
      {

         // ... CaptureSource, VideoBrush initialization 

         //  Initialize the Detector
         arDetector = new CaptureSourceMarkerDetector();
         // Load the marker pattern. It has 16x16 segments and a width of 80 millimeters
         var marker = Marker.LoadFromResource("Marker_SLAR_16x16segments_80width.pat", 16, 16, 80);
         // The perspective projection has the near plane at 1 and the far plane at 4000
         arDetector.Initialize(captureSource, 1, 4000, marker);

         // Attach the AR detection event handler
         // The event is fired if at least one marker was detected
         arDetector.MarkersDetected += (s, me) =>
         {
            // Change to UI thread in order to manipulate the text control's projection
            Dispatcher.BeginInvoke(() =>
            {
               // Calculate the projection matrix
               var dr = me.DetectionResults;
               if (dr.HasResults)
               {
                  // Center at origin of the TextBlock
                  var centerAtOrigin = Matrix3DFactory.CreateTranslation(-Txt.ActualWidth * 0.5, -Txt.ActualHeight * 0.5, 0);                  
                  // Swap the y-axis and scale down by half
                  var scale = Matrix3DFactory.CreateScale(0.5, -0.5, 0.5);
                  // Calculate the complete transformation matrix based on the first detection result
                  var world = centerAtOrigin * scale * dr[0].Transformation;

                  // Calculate the final transformation matrix by using the camera projection matrix from SLARToolkit
                  var vp = Matrix3DFactory.CreateViewportTransformation(Viewport.ActualWidth, Viewport.ActualHeight);
                  var m = Matrix3DFactory.CreateViewportProjection(world, Matrix3D.Identity, arDetector.Projection, vp);

                  // Apply the final transformation matrix to the TextBox
                  Txt.Projection = new Matrix3DProjection { ProjectionMatrix = m };
               }
            });
         };
      }

The final C# code-behind:

using System.Windows;
using System.Windows.Controls;
using System.Windows.Media;

using SLARToolKit;
using System.Windows.Media.Media3D;

namespace SLARToolKitBeginnersGuide
{
   /// <summary>
   /// MainPage of the Beginner's Guide
   /// </summary>
   public partial class MainPage : UserControl
   {
      CaptureSource captureSource;
      CaptureSourceMarkerDetector arDetector;

      public MainPage()
      {
         InitializeComponent();
      }

      private void UserControl_Loaded(object sender, RoutedEventArgs e)
      {
         // Initialize the webcam
         captureSource = new CaptureSource();
         captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();

         // Fill the Viewport Rectangle with the VideoBrush
         var vidBrush = new VideoBrush();
         vidBrush.SetSource(captureSource);
         Viewport.Fill = vidBrush;

         //  Initialize the Detector
         arDetector = new CaptureSourceMarkerDetector();
         // Load the marker pattern. It has 16x16 segments and a width of 80 millimeters
         var marker = Marker.LoadFromResource("Marker_SLAR_16x16segments_80width.pat", 16, 16, 80);
         // The perspective projection has the near plane at 1 and the far plane at 4000
         arDetector.Initialize(captureSource, 1, 4000, marker);

         // Attach the AR detection event handler
         // The event is fired if at least one marker was detected
         arDetector.MarkersDetected += (s, me) =>
         {
            // Change to UI thread in order to manipulate the text control's projection
            Dispatcher.BeginInvoke(() =>
            {
               // Calculate the projection matrix
               var dr = me.DetectionResults;
               if (dr.HasResults)
               {
                  // Center at origin of the TextBlock
                  var centerAtOrigin = Matrix3DFactory.CreateTranslation(-Txt.ActualWidth * 0.5, -Txt.ActualHeight * 0.5, 0);                  
                  // Swap the y-axis and scale down by half
                  var scale = Matrix3DFactory.CreateScale(0.5, -0.5, 0.5);
                  // Calculate the complete transformation matrix based on the first detection result
                  var world = centerAtOrigin * scale * dr[0].Transformation;

                  // Calculate the final transformation matrix by using the camera projection matrix from SLARToolkit
                  var vp = Matrix3DFactory.CreateViewportTransformation(Viewport.ActualWidth, Viewport.ActualHeight);
                  var m = Matrix3DFactory.CreateViewportProjection(world, Matrix3D.Identity, arDetector.Projection, vp);

                  // Apply the final transformation matrix to the TextBox
                  Txt.Projection = new Matrix3DProjection { ProjectionMatrix = m };
               }
            });
         };
      }

      private void Button_Click(object sender, RoutedEventArgs e)
      {
         // Request webcam access and start the capturing
         if (CaptureDeviceConfiguration.RequestDeviceAccess())
         {
            captureSource.Start();
         }
      }
   }
}


10. Print the SLAR marker non-scaled at the original size (80 x 80 mm) and centered for a small white border. See the markers documentation for details and how to make / use a custom marker.


11. Start the application (F5 key), click the Start Fun Button and hold the printed marker in front of the camera and move it around. Make sure the scene is illuminated well without hard shadows.
You should now see how the TextBlock is transformed when you move the marker.

Screenshot

Last edited May 17, 2010 at 9:18 PM by teichgraf, version 37

Comments

edcalderin Jan 2 at 9:34 PM 
How Do I put an object 3D?

jpxgen Oct 3, 2012 at 11:56 PM 
Getting the error "Object Null reference error" while initializing the markers in the step 9
var marker = Marker.LoadFromResource("Marker_SLAR_16x16segments_80width.pat", 16, 16, 80);
Any help could be highly appreciated.

Thanks,
JP

dhavalcharadva Apr 14, 2011 at 3:42 PM 
Really good tool. I want it to use to read text from marker patter and want to do some operation in application, is it possible & how ?

uppanapalli Dec 8, 2010 at 7:35 AM 
Everything is working fine and i am able to see the results. But If we can Improve this to show multiple items like animations and videos etc... that will be great! ... I will try to do it :)

blackcloud22 Jun 1, 2010 at 10:47 PM 
Right! Success! Everything worked out and now I want to continue to improve my knowledge with the tool and contribute to the project! Thank you!