Unit Testing meets Umbraco 4.1

by Shannon Deminick 17. May 2010 14:42

Yup, it's true, unit tests are now part of Umbraco 4.1!

I was starting to fix a few bugs listed on CodePlex for Umbraco and a lot were related to adding foreign keys, primary keys, indexes, etc… in the database. I was quite happy to take up the much needed task of putting some real structure into the underlying table structure but once I started adding the constraints, SQL server started telling me there was problems. Turns out that adding the correct foreign keys and indexes started to expose quite a few bugs in the data layer. These bugs are mostly related to data integrity such as deleting rows from the umbracoNode table but not from the related cmsDataType table when removing an Umbraco Data Type. In light of knowing that there was a lot of testing ahead of me I decided to figure out how to get unit tests working with Umbraco (otherwise testing all of these data constraints was going to be painfully slow and tedious).

So what’s the big deal? Everyone writes unit tests for code! Well, if you’ve used the Umbraco API for anything, you’ll quickly realise that you can’t use any of the Umbraco APIs unless you’re using them in an ASP.Net web context and having a web context in a unit test framework is generally hard to come by without some tricky type mocking. Here’s where Microsoft’s Visual Studio unit testing framework comes in very handy! It’s actually very easy to make a Visual Studio unit test run in the same web context as your website and here’s how:

  • Create a new Unit Test project in Visual Studio
  • You’ll notice that it creates a .testrunconfig file in your Solution Items folder in your solution
  • If you double click this file, it will present you with some options, click on the "Hosts" option:

image

  • As seen above…
    • Select ‘Run in default host’ (I’m actually not sure if this is required :)
    • Choose ASP.NET in the Host Type drop down list
    • Change the URL to test to be the URL that your web application is running under (or if you’re running in IIS, select Run tests in IIS)
    • Choose the path to your web application files in the "Path to Web site" field
    • Enter a "/" in the Web application root (assuming you’re not running in a virtual directory)
    • Close and save the .testrunconfig file
  • Now, in your unit test class, you just need to ensure that your tests are run with this config:

image

Now when you run your unit test, it will run in the context of your web application!

Caution!

A couple of things to consider with this configuration:

  • If you’re automating unit tests using a build server, or similar, this isn’t going to work. This will work on your machine because it’s running under the Cassini context built in to Visual Studio. Build servers aren’t going to automagically be able to start up a Cassini server to run your tests under (well, i’m sure anything is possible, but i don’t think this is going to work out of the box). Running it under IIS could be an option however.
  • Debugging doesn’t work quite the same as normal unit test. You can’t actually just set a break point and hope that it will stop code execution and allow you to debug but fortunately there’s a work around for this. You’ll have to add a user defined break statement in your code if you want to step through it:
System.Diagnostics.Debugger.Break();
  • When you add this, an alert will pop up asking you if you want to debug so all you have to do is click yes and start debugging with another instance of Visual Studio. You can just keep this instance attached to the process and add breakpoints to this instance as much as you like.

Working in teams

As you can tell from the above configuration file, that this setup is specific to my computer which is why I’ve named the .testrunconfig file using my machine name: Shandemvaio. This is useful when working in teams (such as the Umbraco core team). Each member of the team can create their own .testrunconfig file and before running the tests just set the Active Test Run Configuration file to be their own… nice.

The Tests

So far I’ve written some tests for the Document object. I’ve started with this object since it’s probably the most important and is also the highest in the object hierarchy which means that the test written here will also be testing a lot of the base class functionality. We could write unit tests for months and still not have written tests for every circumstance so I’m merely focusing on unit tests that relate to data integrity. And here’s the ones written so far:

image

… and there’s lots more to come!

Categories: .Net | Umbraco

Umbraco 4.1 Benchmarks Part 2 (Back Office Database Queries)

by Shannon Deminick 28. April 2010 17:03

This is part 2 in a series of Umbraco 4.1 benchmarks created by various members of the core team in the lead up to launch. See Part 1 here on request/response peformance in the Umbraco back office.

This benchmark report looks at the data layer improvements in 4.1 by comparing query counts in 4.1 to 4.0.3. Not only has the data layer improved but there’s been significant improvements in the consumption of the data layer API made by many of the 4.1 pages and controls.

The stats below are represented as a percentage of the total calls of 4.0.3 where the number of queries in 4.0.3 are 100% and the number of queries in 4.1 are a percentage in relation to this. These results are based on the procedures listed at the bottom of this post and on averages run over 3 separate trials.

Step 4.0.3 4.1.0
Login 100% 68%
Expand all Content nodes 100% 23%
Edit Home node 100% 49%
Publishing Home node 100% 55%
Edit About Umbraco node 100% 49%
Go to Settings App 100% 100%
Expand Document types tree 100% 100%
Edit Home document type 100% 61%
Save Home document type 100% 67%
Go to Media app 100% 50%
Create new folder labeled ‘Test’ 100% 88%
Create new image under new ‘Test’ folder labeled ‘test1’ 100% 64%
Upload new image file to ‘test1’ and save the node 100% 49%
Go to Content app (and in the case of 4.0.3, expand the tree and select the About Umbraco node since in 4.1 this will already be selected and loaded) 100% 41%
Edit Home node 100% 43%
Add ‘test1’ image to the ‘Text’ WYSIWYG property with the image picker and Publish node 100% 49%
Average of averages above   60%
Complete run through of the above steps 100% 66%

 

So based on averages, Umbraco 4.1 is looking to have around 40% less queries made than 4.0.3!!! Thats HUGE!

The following steps were taken on each trial of the above steps:

  • New instances of both 4.0.3 and 4.1
  • Install CWS package on both instances
  • Log out of both instances
  • Bump web.config for both instances (clear out all data cache)
  • Use SQL Profiler to determine query counts for each step listed above

Also, SQL debugging has been added to 4.1 for MS SQL instances. If you compile the source in Debug mode you can get the SQL command output by adding a trace listener to your web.config. Underneath the configuration node you can add this xml block:

<system.diagnostics>
	<trace autoflush="true">
	  <listeners>
		<add name="SqlListener" 
			type="System.Diagnostics.TextWriterTraceListener" 
			initializeData="trace.log" />
	  </listeners>
	</trace>
</system.diagnostics>

This will create a trace.log file in the root of your web app SQL debugging.

Umbraco 4.1 Benchmarks Part 1 (Do Over)

by Shannon Deminick 19. April 2010 17:06

Doh! Turns out that my first Umbraco 4.1 benchmarks was done on a build of 4.1 that had a couple of bugs in it :( BUT, this turns out to be even better for 4.1 that I previously thought! (plus there’s bugs fixed now :) If you didn’t read the last post, i recommend that you do as it has some insight as to how these stats came to be.

So, without further adieu, here’s the (true) new results:

Test Stats 4.0.3 4.0.3
client cached
4.1 4.1
client cached
Content app Completed Requests 68 7 45 5
Response (KB) 687.05 72.48 382.42 11.61
Edit content
home page
Completed Requests 50 2 34 1
Response (KB) 385.10 47.28 335.02 11.94
Expand all
content nodes
Completed Requests 17 17 16 16
Response (KB) 18.47 18.47 13.96 7.54
TOTALS Completed Requests 135 26 95 22
Response (KB) 1063.62 138.23 731.40 31.09

So to recap, here’s the total average savings in 4.1

  • Without client cache (First run)
    • Number of completed requests: 30% Less
    • Response bandwidth: 31% Less
  • With client cache
    • Number of completed requests: 6% Less
    • Response bandwidth: 78% Less
Categories: .Net | Umbraco

TSQL CASE statement in WHERE clause for NOT IN or IN filter

by Shannon Deminick 16. April 2010 09:58

There’s a ton of articles out there on how to implement a case statement in a WHERE clause but couldn’t find one on how to implement a CASE statement in a WHERE clause that gives you the ability to use a NOT IN or IN filter. I guess the only way to explain this is to use an example, and I am fully aware that the use of this may not be the best practice and is most likely required because of poor database design/implementation but hey, when you inherit code, there’s really no other choice :)

Suppose I have a stored proc that has an optional value:

@OnlyNonExported bit = 0

I want to return all items from MYTRANSACTIONS table if @OnlyNonExported  = 0, but if this value is 1 I want to return all items from MYTRANSACTIONS that have not been tracked in my TRACKEDTRANSACTIONS table. The original theory is to use a NOT IN clause to acheive the latter requirement:

SELECT * FROM mytransactions m 
WHERE mytransactions.id NOT IN (SELECT id FROM trackedtransactions)

So if I wanted to use a case statement for this query, one would think you could do something like this:

SELECT * FROM mytransactions m 
WHERE mytransactions.id NOT IN 
	CASE WHEN @OnlyNonExported = 0 
		THEN  (SELECT -1) 
		ELSE  (SELECT id FROM trackedtransactions) 
	END

But SQL doesn’t like this syntax and it turns out that you cannot use IN or NOT IN conditions with CASE statement in a WHERE clause, you can only use = or != conditions. So how do you achieve the above? Well the answer is even more dodgy that the above:

SELECT * FROM mytransactions m 
WHERE mytransactions.id != 
	CASE WHEN @OnlyNonExported = 0 
		THEN  (SELECT -1) 
		ELSE  COALESCE((SELECT id FROM trackedtransactions t WHERE t.id = m.id), -1)
	END

So basically, when we want to return all transactions, return all rows where the id equals –1 (assuming that your IDs start at 1) and when we want to filter the results based on whether or not these IDs exist in another table, we only return rows who’s IDs don’t match the same ID in the tracked table. BUT if this ID doesn’t exist in the tracked table, then an empty result set is returned and the id won’t be matched against it, so we need a COALESCE function will will return a –1 value if there is an empty result set.

Hopefully you’ll never have to use this but if you do, hope this saves you some headaches :)

Tags:
Categories: .Net

Umbraco 4.1 Benchmarks Part 1

by Shannon Deminick 15. April 2010 18:42

This is the first installment of what will hopefully be many Umbraco benchmark reports created by various members of the core team in the lead up to the launch of Umbraco 4.1. This benchmark report is about the request/response performance of the Umbraco back-office. This compares 4 different configurations: 4.0.3 with browser cache disabled (first run), 4.0.3 with browser cached files, 4.1 with browser cache disabled and 4.1 with browser cached files. These comparisons have been done by using newly installed Umbraco instances with ONLY the CWS package installed. The benchmark results were prepared by using Charles Proxy.

Test Stats 4.0.3 4.0.3
client cached
4.1 4.1
client cached
Content app Completed Requests 68 7 46 6
Response (KB) 687.05 72.48 431.41 32.54
Edit content
home page
Completed Requests 50 2 34 1
Response (KB) 385.10 47.28 343.36 12.07
Expand all
content nodes
Completed Requests 17 17 16 16
Response (KB) 18.47 18.47 13.96 10.85
TOTALS Completed Requests 135 26 96 23
Response (KB) 1063.62 138.23 788.73 55.46

Note: the above is based on <compilation debug=”false”> being set in the web.config. If it is set to true, the compression, combination and minification for both the ClientDependency framework and ScriptManager is not enabled. Also, this is not based on having IIS 7’s dynamic/static compression turned on, these benchmarks are based on Umbraco performing ‘as is ‘ out of the box which will be the same for IIS 6.

Though there’s only 3 tests listed above, these results will be consistent throughout all applications in the Umbraco back office in version 4.1.

The 4.1 difference:

  • In 4.0.3, all ScriptResource calls generated by ScriptManager were not being compressed or minified. This was due to a browser compatibility flag that was set in the base page (this was probably very old code from pre v3!).
  • Script managers in the back-office have the ScriptMode=”release” explicitly set (for minification of ScriptResource.axd)
  • The ClientDependency framework is shipped with 4.1 and all of the back office registers it’s JavaScript and CSS files with this framework. This allows for:
    • Combination, compression, minification of dependencies
    • Rogue script/style detection (for those scripts/styles that weren’t registered with the framework will still get compressed/minified)
    • Compression/minification of specified Mime types, in this case all JSON requests in the back office (namely the tree)
    • Compression/minification of all JavaScript web service proxy classes (‘asmx/js’ requests that are made by registering web services with the ScriptManager
  • Much of the back office client scripting in 4.1 has been completely refactored. Most of the JavaScript has been rewritten and a ton of file cleanup has been done.

Compared to 4.0.3, this is a HUGE difference with some serious performance benefits!

Categories: .Net | Umbraco | ClientDependency

ClientDependency now supporting MVC

by Shannon Deminick 6. April 2010 18:38

I’m please to announce that the ClientDependency framework now supports MVC! It’s very easy to implement using HtmlHelper extension methods. Here’s some quick examples:

Make a view dependent on a CSS file based on a path defined as “Styles”

<% Html.RequiresCss("Content.css", "Styles"); %>

Make a view dependent on jQuery using a full path declaration:

<% Html.RequiresJs("/Js/jquery-1.3.2.min.js"); %>

Rendering the Style blocks and defining a global style path:

<%= Html.RenderCssHere(new BasicPath("Styles", "/Css")) %>

Rendering the Script block (no global script path defined):

<%= Html.RenderJsHere() %>

There’s still a provider model for MVC but it uses a slightly different implementation from Web Forms. The same compositeFiles provider model is used but instead of the fileRegistration provider model that is used in Web Forms, a new mvc renderers provider model is used. A renderer provider is similar to the Web Forms fileRegistration providers but instead of registering the markup in the page using the page life cycle, a renderer provider is used to render out the html block necessary to embed in the page.

All of the functionality that existed in Web Forms exists in MVC. You can make as many views that you want dependent on as many of the same or different client files that you want and the system will still sort by position and priority and remove all duplicate registrations. Rogue scripts & styles still get processed by the composite file provider in MVC. Currently however, if you place user or composite controls on your views that have Client Dependencies tagged with either the control or attribute method used in Web Forms, these will not be registered with the view and output with the renderer. 

MVC pages have been added to the demo project as examples so have a look! You can download the source HERE

For full details and documentation go HERE

Examine hits RC1

by Shannon Deminick 5. April 2010 12:04

I’m happy to announce that Examine and UmbracoExamine have today hit RC1!FileDownload[1]

The Codeplex site also has more extensive documentation about how to get UmbracoExamine up and running within your Umbraco website.

Go, download your copy today.

Categories: .Net | Examine | Umbraco

jQuery fade bug in IE – background transparent PNG image

by Shannon Deminick 26. March 2010 08:20

(I’ve found a ‘solution’ to this… see bottom of post)

Problem

I’ve been trying to figure this out for the whole day today and have finally succumbed to the realization that I’m pretty sure I’ve found a bug in jQuery, even the latest version.

The example link is below. The mark-up is very simple and all that is being done is:

  • Create a box with a single pixel semi transparent PNG image
  • Call either fadeIn or fadeOut on any HTML element on the page… suddenly (and strangely) the box magically has a semi transparent gradient on it!

So if anyone out there know how to fix this, I’m all ears. I’m about to go searching on the net for an alternative fade in/fade out library that doesn’t produce these results but would obviously like to just use the built in jQuery methods.

Click here to see the bug and source! (You’ll have to view in IE 7/8 to see the bug)

Before fade out on the black box:

image

After fade out on the black box:

image

 

VERY STRANGE!!!!

Solution

I knew this had to be an issue with IE’s stupid DXImage transform stuff but didn’t really want to go down the route of fiddling with it but in the end, had to do it. And also had to add a stupid little hack to target IE directly.

Step 1:

Create a IE conditional div wrapper around everything under body:

<!--[if IE]><div id="ieWrap"><![endif]—>

(the rest of the page html goes here)

<!--[if IE]></div><![endif]—>

Step 2:

Target the elements in CSS for IE that have the background PNG transparent image:

#ieWrap .productView .productThumbnail a
{
	background-image: none;
	filter: progid:DXImageTransform.Microsoft.AlphaImageLoader(src='grey_overlay.png',sizingMethod='scale');
}

Click here for solution

And, a day later i found out this isn’t working as expected in IE 7… great… so figured out i had to add: width:100% to the style with the DXImageTransform.

Tags: , ,

ASP.Net Client Dependency Framework RC1 Released!

by Shannon Deminick 18. March 2010 17:13

With the community feedback, bug reports, patches, etc… I’ve managed to find time to upgrade this library to a release candidate status. We do use this framework in many production websites but it was great to hear from other in regards to specific bugs that were found relating to particular environments. These bugs have all been fixed up and this library is looking very stable.

You can download the binaries here.

Better yet, I’ve put together a near complete documentation library on CodePlex here !!

I still think the best way to learn about this project is to download the source code from CodePlex here and have a look at the demo web application included.

Moving forward, the next phase for this library is to add MVC support and another file registration provider called PlaceholderProvider which will give you even more granular control over where dependencies can be rendered in your markup. MVC support should be fairly straight forward and we’ll include a demo project for this as well.

Well definitely be releasing a final version soon after the next Umbraco 4.1 release candidate is released (which will hopefully be fairly soon!)

Happy day! Any and all feedback, bug reports and patches are definitely appreciated!

Umbraco 5th birthday meetup in Sydney @ TheFARM

by Shannon Deminick 12. February 2010 11:39

Come on down to TheFARM to share in some beers and take part in the global Umbraco 5th birthday festivities.

Your hosts will be core Umbraco team developers Shannon Deminick & Aaron Powell, both of whom work for TheFARM (http://www.thefarmdigital.com.au).

The plan is for Shan and Aaron to run a Q&A session with some demo’s of the fun stuff TheFARM has been doing with Umbraco 4.1 and the work they have been doing on this next release.

  • They'll have a look at all of the new features/fixes for 4.1 (are there are TONS)
  • They'll go into a bit more in detail on some of the new things that we've integrated into the core such as LINQ to Umbraco, Umbraco Examine, new controls, enhancements, preview, etc…
  • They’ll show you some of the sites we’ve built and talk through some of the implementation’s with things like Flash

Hopefully, with two of the core team on hand we should be able to answer most questions thrown at us – give us a go!

Once we're out of beers... TO THE PUB!

All of the details, address, etc.. is on the Our Umbraco website. Have a look and RSVP now!

http://our.umbraco.org/events/umbraco-5th-birthday-meetup-in-sydney

 

Just in case you don’t want to click through here’s the event details:

Tuesday, February 16, 2010 - 6:00 PM
Suite 101, 4 - 14 Buckingham st Surry Hills, NSW

Categories: .Net | Flash | Umbraco