Blog Home  Home |  Breeze Home RSS 2.0 Atom 1.0 CDF  
Mick's Breeze Blogs - Biztalk/Sharepoint/... - .NET Developer
Things hard and not so hard....
 Thursday, August 07, 2008

Wow! I had to venture into the 'cave' and solved this problem - talk about a character building experience!

I'm currently building a Mobile BizTalk RFID 1.1 solution for TechEd08 that runs on a PPC with a Kenetics CFUHF Reader.

*** Early Screen Shot *** :) image

So in building out this application the details always bring unforeseen challenges to light:

1) The application houses all the BizTalk RFID pieces (providers, device proxies etc) so registration, and starting/stopping providers/device discovery and applying properties to the device needs to be all taken care of.

2) I built an RFID Mobile Provider for the Kenetics device - I worked with their support engineers solidly for a week to build what I needed. I took a trip down memory lane and have had enough pinvoking to last till Christmas.

3) The app also manages a several local SQLCe databases - one for my app, the others for the operation of BizTalk RFID Mobile locally on the device (mainly for it's OOTB store/forward mechanism).

After weighing up several options in this solution and how to get data to/from the device reliably I decided to go with SqlCe Merge Replication as we needed to push/pull data from several tables and schema changes.

4) Which leads me onto one of the most little known items......

How do I setup SqlCe Merge replication? it's a mine field, change something here and boom over there.

The picture
image

Phase 1:

Forget ISA for the moment. If you can, aim to get replication running in a local environment first (e.g. Local LAN on same network, through VPNs etc)

Getting the SQL bits Setup
Ok - the pieces to the initial puzzle.....

  1. Sql Server Side
    1. Sql Server and it's additional Sql Mobile Replication Bits - download from here.
    2. IIS to expose a replication 'end point' where the remote devices will connect to and replication will take place through. IIS can be separate out onto a different machine.
    3. As in my case, somewhere that the 'snapshot' DB information will live to merge down to the devices. Mine was a UNC share - SQL created this after I completed the Publication wizard.
    4. Installation -You want the SQL Server Compact 3.5 Server Tools installed on BOTH the IIS AND SQL Machines (if these are one and the same, then you only need it once)
      The server tools has two main components - one being the bits that drive IIS and the other being a wizard that configures the exposed virtual directory and sets security onto it.
      If IIS and SQL are on separate machines, the easiest way to go is:
      -
      get SQL to publish the snapshot to a UNC share e.g. \\sqlserver\data
      - On the IIS box, run the Configure Web and Synchronization Wizard (installed with the above server tools) and a later screen will ask you where this data is coming from - simply point to the UNC share.
  2. Mobile Device Side
    1. The equivalent SQL Mobile Replication tools need to be installed (above and beyond just normal SqlCe database components install) - SQL Server Compact 3.5 for Windows Mobile
      *** NOTE: make sure that the bits on both the Mobile + Servers all match ***
  3. Server Side Security - For this let's work backwards, from the publication through to the exposed endpoint.
    1. Publication Security - this is set through the Publication Access List within SQL Mgmnt Studio
      The group in question is the ExhibitorsGroup

      Create a publication within the SQL Management Studio

      image
      (Publication General Properties)

      image
      (Snapshot Properties - note the file location)

      image
      (FTP Snapshot + Internet - I've just used Internet and no IIS server name as this is configured in the Mobile Wizard)

      image
      (Publication access list - I've blanked out sensitive info, but you can see the BETDEV\ExhibitorGroup being manually addded to the list)
      The rest of the publication settings are defaults - for me anyway.
    2. Let's go to the UNC share - = C:\Public\Exhibitor.SqlCE.FileShare
      This is the UNC share that IIS repl component will connect to at the back end.
      Note: the BETDEV\ExhibitorsGroup obviously needs r/w access to this folder.
      image
    3. Let's run the 'Configure Web and Synchronization Wizard' to configure the IIS component.
      (you'll find it off the tools menu after you've installed the Mobile Server Tools from the links above)
      Note: one of the interesting things I found here is that after running the wizard, I normally go a tweak a few things in IIS - directory browsing etc. As a rule of thumb, if you want to change something with the Virtual Directory that is created at the end of this wizard, re-run the wizard to do it!!! :)

      image
      Press next if prompted with the welcome screen note my options here - SQL Mobile and press Next.Cool

      image
      Select the site and Create a Virtual Directory (I'm re-running the wizard so I'm going to select Configure Existing). Press Next.

      image
      I created an alias of SqlCERepl directory and accepted a sub-directory under the SqlMobile dir.
      (you can change this, but looking around the forums it was a source of grief - I could do without :) )

      image
      Here - I selected HTTP and not HTTPS access to the VirtualDirectory (and SQL Service agent).
      I did this as if you remember the diagram at the top of this post, ISA will serve as the HTTPS endpoint and will fwd the request via HTTP to our IIS/SQL box.
      IF you do want to change from HTTP to HTTPS or visa versa - re-run this wizard. Save you about 4 hrs of head banging.
      Click Next when ready.

      image
      On this page - I selected 'Authentication required' and not anonymous. This has something to do with the data that I'm replicating as I'm using a Filter based on 'UserName'. So in my case, the username that the devices connect with will be my differentiator (I looked into using something like 'deviceID' but didn't get too far)
      Click Next.

      image
      Select the type of authentication to be made against IIS - I selected NTLM (basic is fine also - but you need to be mindful that we're using HTTP at this point)
      Quick note on Security: So far, we've got 2 areas that need authentication.
      1) the IIS virtual directory and 2) accessing the actual SQL Publication in the UNC share and SQL Publisher Access List.

      So if the two machines are separated (IIS + Sql), NTLM will no transverse these machines (known as the 'double-hop' problem) so I'm assuming Basic or Kerberos is the safer bet.
      Click Next when ready.

      image
      On the Directory Access Screen note the presence of the ExhibitorsGroup and also this publication is accessing the UNC Share we created earlier.
      Next to continue.

      image
      UNC path specified - here you can see how this could be pointing to this SQL Share sitting on another machine as in the 2 machine hosted scenario.
      Click Next and Finish to see something like:
      image

      You're virtual directory is now configured.
      To test your configuration so far go to:
      /sqlcerepl/sqlcesa35.dll?diag" temp_href="http:///sqlcerepl/sqlcesa35.dll?diag">http://<server>/sqlcerepl/sqlcesa35.dll?diag - diagnostics screen to get something like:
      You should be prompted to login - enter account details that have access.
      image
      This is our fallback screen - next we will configure the ISA component and come back to our test screen to make sure.
      You're done - here. :)
  4. Configure ISA Server
    ISA server will be the bridge between our public SSL access and our internal IIS/SQL Server. We would effectively like ISA to simply route the request and pass it through without to much tampering with our good packets.

    ISA Server is on IP address: IP:Y_Internal
    The Internal Server here is : 10.1.0.191
    The public Interface on the ISA Server is for our purpose known as IP:X_Public
    and it's FQDN is : demo.micks.org (in otherwords - this is the public DNS name that will point to the public interface of your ISA box)

    NOTE: Make sure you have your SSL cert ready - I created an inhouse cert from a standalone cert server.
    You need at least a 'Server Authentication' Certificate to apply within ISA.
    (I'll show you a little trick in the mobile app to get round the fact that the certificate is from a non-trusted Cert. Authority by default)
    The friendly name on the cert should be - 'demo.micks.org' (without the quotes)
    All this keeps SSL happy.

    1. Create a publishing rule in ISA 2006 that will effectively route all requests coming to the public interface to our internal IIS/SQL Server.
    2. Fire up the ISA MMC and create a New Web Server Publishing Rule - I've called this sample rule, "Public to Internal IIS/SQL Repl"

      image
      Click Next when done.
    3. Rule Action - set to Allow
      image
      Next
    4. Publishing Type=Single Web
      image
      Next
    5. Server Connection Security - SSL.This means that SSL is going to be used over the public network.
      image
      Next
    6. On the Internal Publishing Details - I tend to hardcode the IP address in, just to reduce any ambiguity.
      Note the IP address - internally acessible only. 10.x.x.x
      image
      Next
    7. Further settings on the Internal Publishing Details
      NOTE: the option of fwding the original client host headers to the internal IIS/SQL (I found a variety of incomplete  HTTP Header details errors attempting to sync if I cleared this checkbox)

      We also can restrict the access on this rule by specifying the path of /SqlCeRepl/* (this is obviously the Virtual Directory created earlier)
      image
      Next
    8. Fill in your public DNS name - don't worry that the wizard screen is showing http://demo.micks.org and NOT https://demo.micks.org
      image
      Next
    9. Create a listener (if you need to ) as follows:
      (I've modified the screen shot slightly - from my listener)
      Note the ports: 8443 that SSL requests is coming on. You can use 443 if you prefer, I had other things on those ports)
      Also - I setup NO Authentication and replication works. You *could* try setting up Basic Authentication here and using Delegated Authentication (ISA server will login to the IIS/SQL box on your behalf with the inputted security credentials).

      I've also supplied the Certificate here as well (add your cert to the machine store ahead of time)

      A way to test if your auth is going to work - fire up your browser and try /sqlcerepl/sqlcesa35.dll?diag" temp_href="http:///sqlcerepl/sqlcesa35.dll?diag">http://<server>/sqlcerepl/sqlcesa35.dll?diag

      You should be prompted for login details ONLY ONCE. If you need to supply them twice and then you see the diagnostic page, your mobile application replication will fail :-(. Once and once only.
      image
      Next.
    10. Authentication Delegation- we want the client to auth. directly against the backend (routed through ISA of course :) )
      image
      Next.
    11. User Sets - because we don't have authentication here, ISA can't determine users, so All Users is our only option.
      image
      Next.
    12. What a glorious site....almost done......
      image
      Click Finish to complete the wizard.
    13. Right click on the rule just created and select Properties - we need to change the Link Translation to OFF
      image 
      This was the major source of my grief - I kept getting 'HTTP Headers malformed...' ERROR:28035 when trying to sync from the Device - yay!

      I was fortunate to be able to contact a friend of mine Darren Shaffer (Mobile MVP) that explained what was required to be sent back/forth in the headers during the conversation - big thanks there Darren!
    14. You should be able to browse to /sqlcerepl/sqlcesa35.dll?diag" temp_href="https:///sqlcerepl/sqlcesa35.dll?diag">https://<yourserver>/sqlcerepl/sqlcesa35.dll?diag - it should WORK :)
      If not - resolve before moving on. (you may get IE grumbling about the Certificate being invalid if it's an inhouse cert)
  5. Configure the MOBILE replication piece!!!
    1. Make sure you have installed the SQL CE 3.5 Core + Repl CABs at least.
    2. On the mobile device, I tend to have routines to Add and Remove DB Subscriptions as I found that if any publication changes on SQL Server happened - e.g. a field was modified, or a table added/removed from the Publication, then Merge Repl would fail even though it previously was working.

      Easier to Remove the Subscription on the local SQLCE db, and then add it again.

      Note: InternetUrl = " temp_href="https://">https://<yourserver.com>
      Username + pass must be a user that has access to all the bits we configured above. In my case, someone who is a member of the ExhibitorsGroup.

      The code looks like this:
         1:   public void AddReplAndSync()
         2:          {
         3:              //using System.Data.SqlServerCe;
         4:              bool bAddRepl = false;
         5:              try
         6:              {
         7:                  if (DoDBLookup("SELECT count(*) as cRow FROM __sysMergeSubscriptions WHERE Subscriber='ExhibitorSubscription'", "cRow") != "1")
         8:                  {
         9:                      bAddRepl = true;
        10:                  }
        11:              }
        12:              catch 
        13:              {
        14:                  bAddRepl = true;
        15:              }
        16:   
        17:              SqlCeReplication repl = new SqlCeReplication();
        18:              repl.InternetUrl = AppSettings.Settings.ReplServer +  "sqlcesa35.dll";
        19:              repl.InternetLogin = AppSettings.Settings.ReplUser;
        20:              repl.InternetPassword = "XXXXXX";
        21:              
        22:              repl.Publisher = AppSettings.Settings.ReplPublisher;
        23:              repl.PublisherDatabase = AppSettings.Settings.ReplPubDB;
        24:              repl.PublisherSecurityMode = SecurityType.NTAuthentication;
        25:              repl.Publication = AppSettings.Settings.ReplPubName;
        26:              repl.Subscriber = AppSettings.Settings.ReplSubName;
        27:              repl.SubscriberConnectionString = string.Format("DATA SOURCE='{0}'", ESDAL.GetDBPath());
        28:              
        29:              try
        30:              {
        31:                 if (bAddRepl)
        32:                     repl.AddSubscription(AddOption.ExistingDatabase);
        33:                 CloseAllDBConnections();
        34:                 repl.Synchronize();
        35:              }
        36:              catch (SqlCeException e)
        37:              {
        38:                  MessageBox.Show(e.ToString() + e.NativeError.ToString());
        39:              }
        40:              
        41:          }
        42:   
        43:          public void ReplRemove()
        44:          {
        45:              CloseAllDBConnections();
        46:              SqlCeReplication repl = new SqlCeReplication();
        47:              repl.SubscriberConnectionString = string.Format("DATA SOURCE='{0}'", ESDAL.GetDBPath());
        48:              repl.InternetUrl = AppSettings.Settings.ReplServer +  "sqlcesa35.dll";
        49:              repl.InternetLogin = AppSettings.Settings.ReplUser;
        50:              repl.InternetPassword = "XXXXXX";
        51:              repl.Publisher = AppSettings.Settings.ReplPublisher;
        52:              repl.PublisherDatabase = AppSettings.Settings.ReplPubDB;
        53:              repl.PublisherSecurityMode = SecurityType.NTAuthentication;
        54:              repl.Publication = AppSettings.Settings.ReplPubName;
        55:              repl.Subscriber = AppSettings.Settings.ReplSubName;
        56:              try
        57:              {
        58:                  CloseAllDBConnections();
        59:                  repl.DropSubscription(DropOption.LeaveDatabase);
        60:              }
        61:              catch (SqlCeException e)
        62:              {
        63:                  MessageBox.Show(e.ToString() + e.NativeError.ToString());
        64:              }
        65:          }
        66:   
        67:          private void CloseAllDBConnections()
        68:          {
        69:              if ((_dbCon != null) && (_dbCon.State != ConnectionState.Closed))
        70:              {
        71:                  _dbCon.Dispose();
        72:                  _dbCon = null;
        73:                  GC.Collect();
        74:              }
        75:   
        76:          }

Trick to deal with Inhouse generated certificates -
Within your mobile app we create a class that essentially returns True when asked 'Is this Cert. valid?'

Somewhere upon starting up your app - e.g. Form_Load - insert LINE#1 below.

LINE#3 onwards describes the class 'MyCustomSSLPolicy'

   1:  System.Net.ServicePointManager.CertificatePolicy = new MyCustomSSLPolicy();
   2:  ......
   3:  using System;
   4:  using System.Collections.Generic;
   5:  using System.Text;
   6:  using System.Net;
   7:  using System.Security.Cryptography.X509Certificates;
   8:   
   9:  namespace MicksDemos.Utilities
  10:  {
  11:      public class MyCustomSSLPolicy : ICertificatePolicy
  12:      {
  13:          public bool CheckValidationResult(ServicePoint srvPoint,
  14:          X509Certificate certificate, WebRequest request, int certificateProblem)
  15:          {
  16:              return true;
  17:          }
  18:      }
  19:  }



Closing note:

Hope you find this useful - I've done this a few times now and am amazed with the lack of info around this especially through ISA.

If you get any errors around "Can't contact SQL Reconciler..." etc errors - GENERALLY try and rebuild the snapshop server side, then try syncing again.

Nighty night!

Thursday, August 07, 2008 12:37:05 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | RFID | Tips  | 
 Wednesday, August 06, 2008

Folks - fellow MVP Richard Seroter has written a VERY comprehensive series around this very topic including the new BizTalk Adapter Pack V1.0 (V2.0 is in Beta at the moment).

Over 20+ thousand words + 178 screen shots - all for the love of BizTalk/WCF.

Complete with Source Code!!!

What a champion series - I'm looking forward to in tucking into some of his great material!

The BizTalk community is in debt to you Richard - well done!!!

SERIES SUMMARY FOUND HERE

Wednesday, August 06, 2008 11:06:02 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [2]   .NET Developer | BizTalk | Insights | Training  | 
 Monday, July 21, 2008

This one came from Paolo Salvatori (a senior PM within the MS Connected Systems Division Team... I know a bit of a mouthful) whom has gotten in touch with his creative side and drawn a picture for all us common folk :) - well done Paolo.

The scenario is - a Request/Response Port is published at the 'front end', goes through BizTalk and the work is done by a backend system that operates via a One-Way Send and BTS gets the response via another One-Way Receive.

The thing I like about Paolo's piece of work is that he shows all the Message Context Properties required to be set by BizTalk for message correlation.
Which makes this a Messaging Only Solution and NO Orchestrations required!!!! (how cool)

 

BizTalk Request Response Port

 

Click on the image to enlarge......one day I'll get Silverlight Zoom Composer control running for these.... :)

Monday, July 21, 2008 7:04:00 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [2]   .NET Developer | BizTalk | Insights  | 
 Sunday, July 13, 2008

After the more than normal pain in getting this done for my previous post, I decided to post the fruits of my labour (not labor that my wife tells me about watermellons and men wouldn't know the first thing about birth....I'm not about to do the pepsi challenge on that :))

- this is a stock standard Web.Config for a MOSS install NOT a plain WSS install (there's about 3 lines different from a plain WSS install to a MOSS install - mainly anything that references SharePoint.Publishing....)

Grab this and these are the changes that WORK!

Enjoy!

Cheers,

Mick.

Sunday, July 13, 2008 10:38:41 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | MOSS | Silverlight  | 

"Could not load file or assembly 'System.Web.Silverlight, Version=2.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies."

You're starting to Roll your sleeves up and get dirty with Silverlight 2 Beta 2, load up some of the Silverlight Blueprint for SharePoint Samples, run the installer (it's great that most of these examples have an installer) and Boom! you get the above error!!!

Here's a list I've compiled to get Silverlight working - I'm currently running this on Win2008 IIS7

(1) Install WSS SP1/MOSS SP1 on your SharePoint box if you haven't already. You need the SP1 to support .NET 3.5 calls through SharePoint - my guess is that these tell SharePoint not to intercept the calls and let them go to their rightful owners.

(2) Create a 'dummy' site collection on a test Web Application - e.g. http://localhost:81 - This is so you can see all the changes to the web.config that are made through the installation process, in isolation. By keeping this separate to your usual web.config, you'll be able to merge changes at a later date.

(3) Install the Silverlight 2 Beta 2 runtime and other developer bits - From http://Silverlight.net - VS2008 Developer Bits and just the runtime if you want from here

(4) Do one installation of a Silverlight for BluePrint Sample - the installer creates a 'virtual directory' under your Web Site called ClientBin where the various Silverlight 2 files go (*.js, *.XAP). This is a handy install so you can see what the directory execution settings are required to make this work through SharePoint. i.e. Execute permissions only. Take note of this directory.

(5) Add a IIS MIME type - With Silverlight 2 beta 2 - there is a new file type added which is a *.XAP file type. IIS by default doesn't know how to encode/translate or send these files down over the wire.
Add a mime type of: Extention: xap Mime Type: application/x-silverlight-app to your IIS Test Web Site

(6) Make Web.Config changes - there's a whole series of Web.Config changes to be made to your SharePoint Web Application to support AJAX/.NET 3.5 and now Silverlight.... fortunately other hard working folks have done this for you!!!! :) Bless their cotton socks! - grab the Feature that makes the modifications from here (** NB: you want the 3.5 config feature)

You're almost done........ :)

(7) EXCEPT for the error above!!! After much inspection of your system, you'll realise that you *don't* have that DLL (on a clean install). The Silverlight Ninja will know that this is from Silverlight 2 Beta 1 and not found in the Beta 2 kits!! Yay team!

The System.Web.Silverlight.dll is found in the Silverlight 2 beta 1 SDK - so download that puppy, extract out the DLL and either GAC it, or add it to your BIN directory on your SharePoint site. (I added it to my BIN directory - as I reckon when SL2 is released, this problem would have been resolved)
(**UPDATED: Due to how painful that was, I decided to package up the DLL for you - HERE**)

Here are the Compiled Files - FOR SL2 BETA 2 - they WORK!! :-)

(I grabbed the Blueprint Hello World Web Part and updated to work)

1. Silverlight Web Part DLL

2. Silverlight *.XAP updated for Beta 2, copy straight to the *sub-directory* under your client BIN

3. Sample SharePoint Web.Config with all the changes! :-)

Sunday, July 13, 2008 9:08:09 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | MOSS | Silverlight  | 
 Thursday, July 10, 2008

One of the handiest tools I've used in the last year - http://technet.microsoft.com/en-us/sysinternals/bb897434.aspx

If you're presenting, even just showing your code, screen to colleagues then this is superb

Thursday, July 10, 2008 8:50:48 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | Other | Tips  | 
 Thursday, July 03, 2008

You'll get this error when using WCF/IIS and host headers.....fortunately a fellow colleague Paul Glavich figured it out!!! Well done Paul! (It involves an IIS reshuffle, you may be able to do something within a custom WCF Binding.)

Remember: There is a limit on the number of IIS Websites you can have on a single machine.

Thursday, July 03, 2008 7:51:38 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | BizTalk  | 
 Tuesday, July 01, 2008

You might be wondering what do all these guys have in common....good question.... :-)

We're currently building an RFID enabled System where complex processes are handled by BizTalk Server, and data being pushed down to Silverlight V2.0 clients via a WCF Silverlight 'Eventing System' (which really is polling under the hood, but to us in developer land - it's cool and it's Events)

Scotty has the full write up of some of his learning experiences through this - well done Scotty, he's been in that place where there are no manuals, no documentation, no previous code, just a gut feel and a compass to sail the seas.

We demo-ed the system at our last user group (or more over used them a guinea pigs :)

Token Screen shot: (we've associated tags with people information and this is what is displayed when TagReadEvents are captured. We need a little work to avoid being underneath or on top of a previous animation)

FULL DETAILS HERE

Artists impression!
Tuesday, July 01, 2008 9:55:17 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | BizTalk | Insights | RFID | Silverlight  | 
 Saturday, June 07, 2008

http://blogs.msdn.com/webdevtools/Default.aspx

Some pretty cool features as I've previously posted

From my perspective I'm particularly interested in the supported 'WCF Dual HTTP Binding'.......more on that later :)

(updated**)

SILVERLIGHT 2.0 BETA 2 SDK is now available!!!!

Grab it here from the Getting Started section

Also get the videos, hands on labs, training material from HERE

Saturday, June 07, 2008 10:50:21 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | General | Silverlight  | 

Jesse has written a great little article on creating a popup control in Silverlight 2.0.

Nice way of doing it - I was also thinking, that quite simply you could also set the ZIndex of the element to a positive value.

Enjoy.

Saturday, June 07, 2008 10:40:30 AM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | Silverlight  | 
 Friday, June 06, 2008

I've started developing a RFID Mobile application on a Windows Mobile device - which is pretty cool.

I decided to use VS2008 and all worked well until deployment.
"Unable to load System.Data.SqlServerCe.dll ....Version=3.0.xxxxx " as the version OOTB with VS2008 is v3.5

So I grabbed SQL CE Mobile 3.0 and copied over and installed the sqlce30.wce5.armv4i.CAB and all is good!

Friday, June 06, 2008 1:31:18 PM (AUS Eastern Standard Time, UTC+10:00)  #    Comments [0]   .NET Developer | RFID  | 
 Sunday, J