I am asked all of the time about the performance of managed vs. unmanaged code and "how much slower is it?". This is one of the questions I am going to attempt to answer using experimentation. In this post I'll talk about some of the theory and make some predictions (I haven't written any code yet) and we'll see how close the theory matches the experiments.
Managed Code Defined
Managed code is somewhat an misunderstood concept. Managed code is simply code that targets the CLR runtime. The CLR runtime is a bit more complex to explain and people often see the tem "virtual machine" and misunderstand the meaning of that statement. When understanding the flow of execution for a managed application there is a three-step process. First, the code is compiled to MSIL (Microsoft Intermediate Language). This could be any code, C#, VB.NET, etc. All Managed code compiles initially to MSIL. MSIL is a low-level language like assembly. It is often described as "Object Oriented Assembly" and rather than using registers you act upon memory in a highly optimized stack called the Evaluation Stack. The beauty of IL is that is is low-level enough to be a very fast JIT (Just In Time) compile to native code, but the optimizations on a per-CPU basis can still be achieved in this final compilation step. CPU optimizations like SSE2, SSE3, etc. that can greatly speed up code execution can be "given" to the users of managed code for free. This could also include other optimizations such as GPU optimizations.
When you run a piece of managed code it will normally take the first bit of execution to covert the MSIL to Native code specifically optimized for your platform. This is known as JIT compilation usually happens very quickly and most people don't even realize that is what is happening every time they launch their application. Of course, if your code is large, complex and has many dependencies you may be able to shave valuable milliseconds or in some cases even seconds by using ngen on your code. Ngen is a tool that ships with the .NET framework that does the same compilation from IL to Native that the JIT compiler does but also keeps a copy of the native image that is generated. This way a managed application is loaded much the same way a native application is loaded -- without JIT'ing.
Figure 1 :: Managed Code Execution Lifecycle
So when you hear that .NET code is run in a virtual machine what they refer to is that you do not have to program to the specifics of a CPU; In that way the code is virtualized. There are also services provided to you such as Memory Management, Thread Management, Exception Handling, Garbage Collection, and Security. However, in my opinion the term virtual machine is a pretty poor fit. These services all run side-by-side under the same process and even the same App Domain. Therefore these services can be better thought of as a standard code template that gets compiled into every application rather than anything that is virtualized.
On the Left - Managed Code
Managed code has some advantages! Not the least of which is productivity. Lets review some areas where managed code actually has an advantage!
CPU Optimizations
As mentioned earlier, Managed Code is JIT compiled to target the specific platform in which you are running. CPU optimizations can have a big advantage in performance! While Native code can also take advantage of CPU optimizations you have to make a trade-off. You have to either ship your code without such optimizations enabled for fear that someone without one of these will try to run your code OR you have to build some code and logic around working with or without each optimization you plan to target.
Little is known, however, about the optimizations performed during JIT compilation. Also, most modern CPU's are likely to have a base set of the most common CPU Optimizations so this argument gets somewhat weaker if there are few differences between chips.
Managed Memory
Managed code has an awesome memory manager! Suppose we have an array of integers like int[]. This is a data structure of contiguous integers. Some of the advantages of using such a structure is that it is very quick to access. You can use a syntax like MyInts[5] for access that is nearly as quick as referencing a local variable. Internally pointer arithmetic is taking place where sizeof(int) * index + MyInts& = MyInts[5]; or Θ(1) which makes this the fastest method of accessing dynamically allocated memory. The down-side to this stricture is adding elements to this array is extremely expensive! Unlike dynamically-linked structures in order to add an element to this array usually requires the allocation of a completely new, bigger array, and the copying of all of the elements from the old array to the new one. This has the complexity of Θ(n). This is extremely expensive for a single add operation!
This is where managed code has another advantage! Because there are no pointers, only system-managed references, it is possible for the memory manager to simply expand the array construct and anything in the way can be safely moved without harm to the execution of the application. The worst case scenario for this operation is now a Θ(1) or that there is a constant cost for such an operation regardless of the size of the current structure. So even in a worst case scenario we have the best possible performance. This is good news for immutable data structures!
Another memory-related advantage managed code has over native code is that it takes advantage of the memory vs. time tradeoff. Basically all .NET applications consume more OS memory than they need. There is some complex memory algorithm at the heart of this that tries to minimize calls to the OS for more memory and that such calls will gain larger blocks of memory. Calls to the OS for more memory are very expensive whereas calls to the Memory Manager in Managed code is extremely fast. Therefore, in theory, we should be able to dynamically create objects substantially faster in a managed language than a native one.
Some of the other advantages to managed memory are the virtual non-existence of memory leaks, and automatic memory defragmentation. And while the last of this list is somewhat controversial, I list it here as an advantage. That is the garbage collector. The garbage collector is the nebulous cloud that hangs over most C/C++ developers looking to write managed code. They have been taught to manage memory themselves and do not like the idea of giving that control up to a complicated set of algorithms. They all ask the same thing -- what happens if garbage collection is triggered at an inopportune time? The answer to that question is a little complicated. But basically the garbage collection is very fast -- usually pausing execution for less than a few milliseconds. Also, because the GC operates on many objects at once it can be a more efficient and less error-prone than self-styled memory management. Basically, the GC is not going to cause you any grief unless you do something stupid like leave your streams open (the only thing I can think of in Managed code that will cause a 'leak') and will mostly be a great burden off of your shoulders!
Security
One of the major reasons for the push for Managed Code was security! Managed code can make certain guarantees about it's vulnerabilities to attack. Even if an exploit is found (which very few, if any have been) then the CLR can usually be quickly patched to provide protection to ALL .net assemblies. I believe this is one of the reasons Microsoft prefers you to allow the JIT to happen for each execution. Part of these guarantees is type safety and bounds checking which are very common exploits for overflow attacks.
If you choose to sign your assemblies you are no longer venerable to "dll replacement" attacks as the CLR will verify the signature of the called assembly. Additionally, with the GAC (Global Assembly Cache) different versions of the same assembly (dll) can live harmoniously side-by-side thus eliminating the infamous "dll hell".
On the Right - Unmanaged Code
Native code enjoys the legacy and reputation of performance. This style of programming is considered "metal to metal" indicating you have complete control over the hardware of the system. These applications are lean and efficient and have no "magic" services which is going to make this section very lean! They use, in general, very little memory compared to their managed counterparts and have the ability to use memory in ways the managed code doesn't like or won't allow.
Native code allows "unsafe" type casting which can result in better performance especially in cases where .net would employ boxing / un-boxing techniques which are very costly. Because nothing is going to happen without you making it happen you should end up with faster code. In smaller applications the memory management could be overkill. You also have much more control over the lifetime of an object. Rather than waiting for a GC to collect the memory at some unknown time you explicitly delete objects and the memory is reclaimed immediately.
There isn't much to say about Native code and I think that's why people are more comfortable with it's performance.
The Matchup
I hope to be able to write the following tests in C# for managed code and C++ and/or Delphi for unmanaged code. I will try to post the code for each "round" and am very open to criticism on fairness. In some respects this is a little bit like comparing apples to oranges but that doesn't mean they can't compete!
Rounds:
- Round 1 : Theoretical (this round)
- Round 2 : Computational (bit manipulation, looping, adding, subtracting, searching, sorting, etc.)
- Round 3 : Dynamic Memory (object creation, array resizing, memory allocation, memory de-allocation, etc.)
- Round 4 : Windows Forms & Messages (dynamic creation of windows, buttons, etc.)
- Round 5 : IO (File System Access, Network Streams, etc.)
The Prediction
Based on the theory I would say that after a managed assembly has been loaded it should execute faster than it's native counterpart. I base this idea mostly on the memory management provided to managed code. Though I would be a little surprised if Managed Code were to win a head-to-head challenge. I believe that the end result will be native code will perform faster than managed code but by a statistically insignificant amount.
I officially call this round: Winner - Managed Code (by decision).
Round By Round Predictions:
- Round 1 : Won by Decision: Managed Code
- Round 2 : Win by Native Code
- Round 3 : Win by Managed Code
- Round 4 : Tie (1 point each)
- Round 5 : Win by Native Code
References:
Pat is an excellent presenter and a friend. I am also excited about SQL Server and Pizza so this meeting was a winning combination for me! It was also nice to see some familiar faces and had a small but decent turnout.
Environment
The meeting started late (as expected with almost anything I attend now days) but was forgivable due to unexpected traffic situations. I personally arrived 5 min's late and was there in enough time to help setup. The pizza came pretty late in the meeting which was a little annoying because I usually eat well before 7:30pm. Also the drinks were all caffeinated (which is usually fine for programmers) but I like the non-caffeinated sort. Also, I liked in the past how those rooms were setup with tables and chairs rather than just tables. I usually like to take notes on my laptop but find the literal term "laptop" rather unworkable.
Microsoft SQL Server 2008
The first question almost asked universally is "when is it going to come out"? Although none of us had any insight into Microsoft it was generally agreed that we had heard 4th quarter 2008. Perhaps even in time for Microsoft's PDC event. Also RC0 is currently available for download.
New features in SQL 2008 include:
- New data types for Date and Time with enhanced precision
- Compressed Backups (YAY)
- New MERGE keyword for simultaneous Insert Update & Delete
- New Table Variables allow you to pass arrays of objects into procs reducing round-trips
- New Code Insight for the SSMS studio
- New Transparent Data Encryption provides more control over encryption in the database
- New Change Detect Capture (CDC) functionality for audit logging
- New Data compression for compressed tables
- New Policy-based management to enforce the stupid naming standards people sometimes put into a database (ughh!)
- New Hierarchy ID field for Parent / Child / Grandchild, etc. Data
- New FileStream Data (remains of WinFS) allows large amounts of binary data to be stored with (but not really in) the database
- New Large User-Defined Types overcomes the 8KB limit for UDT's
- New Spatial Data Types to store lat/long style data
- New Grouping Sets allow multiple GROUP BY statements
- Improvements to SQL Server Reporting Services
We did not talk about every point in that list, spending most of our time split between the new data types and their capabilities and the new MERGE statement. We also had a fun discussion about http://www.sqldumbass.com/ and all of the 'unfortunate' database architectures we have encountered.
Summary
It was a really fun meeting. It was a small crowd with an open atmosphere where we felt like we could ask questions. Pat is more than knowledgeable about such questions and did a great job answering some of those. Of course some of his answers left us scratching our heads as we can't follow a real DBA that deeply!
Links
NUNUG Site: http://groups.google.com/group/nunug
Content Download: 2008DevPresentToNorthernNET.rar
Pat Wright Blog: http://insanesql.blogspot.com/
Have you ever noticed that with IE7 you can only download 2 files from any given domain at any given time? It's actually slightly worse than that, you may only have 2 connections total, including a connection to request web pages. That means if you are downloading two things from a single domain you are unable to browse! Much to my angst this is actually a W3 standard!
Basically it's a quick registry fix. If you are comfortable making changes to your registry this will be really quick!
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings]
"MaxConnectionsPerServer"=dword:00000032
"MaxConnectionsPer1_0Server"=dword:00000032
You can also download the .reg file here
Environment
I have to admit, I was very excited when I saw this was going to be a topic for the users group meeting. I am very interested in hardware! I remember building my own connector to the computer to communicate with my calculator (TI-86). I always wondered how it worked 
so when I saw the topic I was very excited to might learn some of this stuff.
It has been quite a while since I've been to a users group meeting here, but I can never find where we meet. I don't remember what floor or what room and there are never any posters or signs. The meeting started 15 min's late because the presenter was caught in traffic! Also, the wireless at Neumont is always locked down and it really sucks not having Internet when we're group programming and need to know the format string for a DateTime.TryParseExact! Pizza and drinks were plentiful and that is always nice!
The presenter started out apologizing for poor content, usually not a good sign! However he really did seem to know his stuff and came prepared to amaze us with his gobs of cool hardware. He is kind of a quiet talker and while I was near the back of the room I was still plenty close to be able to hear and I had to really strain to hear. If he didn't have code on the screen most of the time I probably wouldn't have gotten much out of it.
I was a little annoyed that we had to wade through the code creation process. Although this is something I like to do in my presentations I only apply it in cases where the topic I am presenting is new. Parsing strings in C# is something we all do all of the time and it was annoying to sit around for over an hour waiting for the code to be created. I was really disappointed when that resulted in skipping Ethernet and Wireless connectivity. I didn't mind so much that the final application wasn't written. I didn't actually even expect anything polished as this is a users group meeting. About two minuets into looking through the code he did have I was satisfied and we could have moved on from there.
Presenter: Josh Perry - 6bit Inc.
http://www.6bit.com/
Sponsor: MindCenter sponsor
http://www.MindCenter.net
The Serial Port
- RS (Recommended Standard)-232C
- Defined in 1969
- DTE (Data Terminal Equipment) - Client
- DCE (Data Circuit-terminating Equipment) - Server
Physical
- Computers wired as DTE
- Devices (modems, etc..) wired as DCE (usually)
- DE-9 most common connector
- DV-25 (25 pin serial connector)
Pin-outs (DE-9)
- RX - 2
- TX - 3
- GND - 5
- RX on DTE goes to TX on DCE
- TX on DTE goes to RX on DCE
- NULL-Modem
Serial - Electrical
- +12v to -12v swing; +12v = 1 - 12v = 0
- Single ended communication, common ground
- Small and embedded systems need level converters and inverters to go from logic to serial levels
- Oscilloscope trace of serial communication
Serial - Protocol
- Time-based sampling
- Baud rate is the frequency of bytes
- Bit rate is 8 * baud + 1 (start bit) + stop bits + Parity
- Most common is 8 bit bytes, no parity, and 1 stop bit
- Baud varies a lot, but 9600 and 115500 are popular
- With only RX, TX, and ground; flow control is none or XON-XOFF. None is most common, XON-XOFF causes problems with binary communications.
Byes are send LSB first if you scope a serial connection under hyper terminal.
We are implementing the NMEA protocol which is a standard maritime protocol that was first implemented for boats. It is an ASCII protocol. Each sentence is 1 line terminated with a CRLF. Comma delimited with each of the values.
Atmel chips are a good way to get started.
The .net microframework chips allow you to program chips using the .net framework.
Links
Every company I consult with invariably has their own "security" assembly and they all have a hard-coded encryption key with the IV and the method to decrypt is right next to the method to encrypt. This is what I call marginal protection. Yes, it's encrypted and will probably get a security auditor off of your back but don't be fooled into thinking that you are protected! A similar thing is done with information in the database, but I'll cover how to do this on an upcoming post.
Why aren't you protected? The answer to this question is actually quite simple. If an attacker has access to download your web.config file (say, they brute forced a password on the FTP server) then there is nothing stopping them from downloading the your Security.dll which is responsible for decrypting the password. Once they have that library it's seconds, not minuets, before they have got the password.
One possible work around is to encrypt configuration sections of your web.config file using DPAPI as outlined in this MSDN How-to. This is immune to the download attack because the DPAPI uses encryption that is based on a machine or a user. Even if someone was able to download your web.config they would effectively have no way to decrypt that information.
What happens, though, if the attacker has the ability to upload files? Well, in theory, they may be able to grab that configuration in code which will, of course, be decrypted before it is returned. Ahh, but they don't even know what the name of the connection string (in the case of databases) is because the entire section was encrypted. However, they could guess it or get it from other code. By the way, you really shouldn't deploy the .cs files to production anyway; you should use the "publish website" option with the setting to not allow the site to be updated. If you follow all of the standards pretty closely your in good shape. Another great idea is to use Integrated Authentication for database access -- that way there is no password to steal!
The How to outlines 3 basic steps summarized below:
- Identify the configuration sections to be encrypted
- You may only encrypt the following:
<appSettings>. This section contains custom application settings.
<connectionStrings>. This section contains connection strings.
<identity>. This section can contain impersonation credentials.
<sessionState>. The section contains the connection string for the out-of-process session state provider.
- Choose Machine or User store
- Use Machine store if this is a dedicated server with no other applications running on it or you want to be able to share this information with other applications running on this machine.
- Use User store if the above does not match your situation and in a scenario in which the user has limited access to the server.
- Encrypt your configuration file data
- To encrypt using Machine Store, run the following command from a .NET command prompt:
aspnet_regiis.exe -pef "{ConfigSectionName}" {PhysicalDirectory} –prov "DataProtectionConfigurationProvider"
OR
aspnet_regiis.exe -pef "{ConfigSectionName}" -app "/{VirtualDirectory}" –prov "DataProtectionConfigurationProvider"
- To encrypt using User Store:
Add the following section to your configuration file:
<configProtectedData>
<providers>
<add useMachineProtection="false" keyEntropy=""
name="MyUserDataProtectionConfigurationProvider"
type="System.Configuration.DpapiProtectedConfigurationProvider,
System.Configuration, Version=2.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a" />
</providers>
</configProtectedData>Open a command prompt using the user you plan to encrypt the file with. To do so, Right click on the Command Prompts shortcut, right click -> Run As. Or use the following command:
Runas /profile /user:domain\user cmd
Run the following command:
Aspnet_regiis -pe "connectionStrings" -app "/{VirtualDirectory}" -prov "MyUserDataProtectionConfigurationProvider"
It really is that simple! The great thing is that we don't have to do anything special in development to benefit from the encryption of the configuration sections.
References:
http://msdn.microsoft.com/en-us/library/ms998280.aspx
I am not a gamer! I would get killed playing anything modern! I was never very good at the games I did play (back in the 90's) like Marathon or StarCraft. I did like games before that but they were mostly DOS based games. I loved Cyberbox, a little game where you try to navigate your dot from one side of the screen to the other pushing blocks along the way. A couple of years ago I was introduced to Line Rider. A really simple game where you "draw" a track for our sledding hero to ride. A simple premise but a fun one! You can spend literally hours without realizing it just designing the most awesome ever track!
LineRider has gone Silverlight! It's been greatly enhanced with the new features of Silverlight and is pretty awesome. I am hoping that more and more Silverlight is adopted and ubiquity increases to match or exceed Flash. I know of no such place that keeps track of these types of statistics but it will be interesting in a year from now how much Silverlight will penetrate the market. It seems that they should be able to make it available as an update via Windows Update like the .net frameworks, but reaching a growing OS X audience may be more difficult. Hopefully there are plans in the works to get Silverlight on that platform.
If I can figure out how to post a capture video from Line Rider, I'll post it here.
Nathan Zaugg
Have you ever needed to add a line to a config file like:
<type="Config.RoleService, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
These fully qualified types are not easy to figure out. The syntax is a bit confusing and the PublicKeyToken is hard to get a look at. This has been an issue -- until now!
GetAssemblyName App
I created a quick little app that will load an assembly you select and show you all of the AQN (Assembly Qualified Names) for each class contained in the assembly. Notice the cool glass look? I'll post information about that in a later blog post.
Quick Instructions:
- Run the app
- Press the "Select Assembly" button
- Browse to the assembly you want to get the AQN for and press Open
- The window will fill up with the assembly full name and the AQN for each type found in the assembly
- Double Click the line to copy that class's AQN to the clipboard
Downloads:
Naming conventions are like arm pits. Everyone has them and they all stink! Well, at least that's the perspective of pretty much every developer an DBA alike. I will present my own personal philosophy for naming conventions on databases and hopefully spawn some discussion in the process.
Basic Principles
Consistency
As annoying as certain standards are (such as putting tbl_ before everything) it is more annoying and more difficult when there are no conventions or mixed conventions. Being able to reliably predict the schema once the basic relational structure is understood is key to productivity. Therefore, even if you get stuck with standards you disagree with, so long as they are consistent they will be much better than the alternative. Unless you get to make the decision, my guess is that there are going to be some conventions that you do not agree with.
Abbreviations
It is a good idea to abbreviate, when appropriate, in the naming of objects in your database. It may be a good idea to have a list of abbreviations that you plan to use in the database as part of your data dictionary. However, if there is not a good, clear abbreviation for an object, don't make one up. When in doubt, spell it out! Especially with SQL Server where you don't have the pesky 30 char limit for tables and columns like Oracle.
Identities
Every table should have an Identity as it's primary key! Sometime, in a future blog post I will explain why this is so critical, but suffice it to say that any table that does not have a primary key is considered by SQL Server a "heap". If you are using something other than an Identity column for the primary key you better have a really compelling reason because it will cause major performance problems. THERE IS NO SUCH THING AS A NATURAL KEY AND THEY SHOULD NEVER BE USED IN PLACE OF AN IDENTITY! So always use a surrogate key approach, even with join tables.
Security
I believe that with a good data layer like Linq to SQL the need for relegating all database access through stored procedures. While it does remove some of the service area for venerability and bugs robust solutions like Linq to SQL are very limited by this approach. You should grant specific access to tables and procs by user. A good approach can be found on another one of my blog posts.
Object Naming
Table Names
- If you are running a database that preserves case (like SQL Server) tables should have no prefixes and should not contain underscores "_" unless it is a join table. Table names should also be Pascal Cased. If you are running a database that makes all tables upper case (like Oracle) then you have little choice but to use underscores everywhere.
- Avoid pluralizing table names (User vs Users). This is a good idea for two reasons, first it can be confusing when doing the keys. Do we use UserID or UsersID? Second not all tables pluralize well (Addresses) so avoiding any plural names will keep it consistent. It you are using Linq to SQL the designer will pluralize for you automatically.
- Join tables should the two or three tables that they are joining together as part of the name seperated by underscores. (ex: User_Address, User_Order). Although they are many-to-many relationship see if you can find a principle table. Users have orders, orders do not have users, therefore the User table comes first in the name.
Column Names
- Name the Primary Key Identity column the table name with ID. (ex: UserID) With the possible exception of join tables, in which case just name the Identity ID.
- Use Pascal casing (ex: EachWordsStartsCapolatized)
- Do not use the table name as part of the column name. If this is a shipping table don't name your columns ShippingAddress, just name it Address.
- Do not prefix column names with the type (ex: strUserName). It makes the database much more difficult to work with.
- Use the correct data types. Always use nvarchar types (unicode) rather than varchar types. This avoids substantial complexity if you are ever requried to store non latin-based data! Trust me, you do not want to have to deal with code pages in the database! Also, use Date fields for dates, bit fields for boolean, etc.
- Don't make every column nullable! Think through what data is absolutely required. If you want to hold "partally complete records" then I would suggest a different table or different "staging" database.
- Don't make a bit field nullable unless you have a great reason!
- Try to include a TimeStamp column if you think you may have to worry about concurrency.
- Don't prefix with anything.
Constraint & Index Names
- Name your constraints and indexes. With the exception of foreign key constraints they are not automatically assigned meaningful names.
- Don't use prefixes and make light use of underscores.
Stored Procedure Names
- Don't prefix your stored procedures! People used to prefix them with "sp" because existing procs in the database use this convention. It has been presumed that sp stands for system procedure and it wouldn't make any sense to use that. Seriously, prefixes are not very helpful in the database!
- The first part of the name of a proc should be the table name it works upon (ex: User_Insert). If the proc works on multiple tables try to give it the name of the portion of the database this proc deals with. For example, if it's a proc that the invoicing system uses it would be acceptable to name it Invoicing_Update, for example.
- Don't generate procs for simple Insert, Update, Delete, and Select unless you have a policy in place for accessing data exclusively from procs.
- Don't create any stored procedure you don't need or plan to immediately use. At some point you will change the schema and you won't update procs your not using. Someone may eventually want to use that proc later only to find it broken.
- The verb in the naming convention does not have to be relegated to "Insert, Update, Delete, Select". It should say what it does. Just be careful that if there is another procedure that does this same thing to another table that the verbs are named the same.
- You can add additional information to the proc name to help distinguish it from others. (ex: User_Select_ByDate, User_Select_ByState)
- Don't use a prefix for arguments (ex: @ArgUserID). In my experience they don't help at all and are quite annoying!
Tips & Tricks
SQL Server 2008 has a policy manager that can help create and enforce policies like naming conventions! Regardless of using SQL 2008 be sure to keep a Data Dictionary of your database! The database is the heart and soul of your business processes and should be well documented! There is nothing worse than an unclean database!
Nathan Zaugg
These past two weeks have been very exciting for me. I have gotten to be involved in some R&D for one of the companies that I consult for. I LOVE R&D! There is always a better way to do things and poking your head out from the sand every once in a while can be very beneficial!
Okay, so here is the story. You want to have auditing so you can log the user responsible for the change. It follows that you simply connect with that users credentials and now you have a great audit log! The problem is that if you have thousands of users (or maybe even less) you are going to start to experience a large number of connections on the server. [Image 1] This is because each user has their own connection pool that, even if it is going through a service, cannot be shared with any other user. A large number of connections is starting to really slow down your database so you decide to create a generic user account for the service. The problem is now our audit log will only show the service account as the person responsible for the change! [Figure 2]
Image 1
Image 2
So you have two ways in which you can fix this. First you can mandate that all changes to the data must happen through stored procedures. If we make sure that every stored procedure passes the user who is responsible for the DML changes then we can add our own audit records. The upside is that not only can we take full advantage of connection pooling and security is better using procs. The downside is that this can be intensive and the change log probably cannot be driven by triggers and we may have to come up with a complex and fallible process.
Alternatively, you can use a basic service account for the connection and connection pool and run the SQL 2005 / 2008 "EXECUTE AS LOGIN" command before any other DML statement. [Image 3] This is called User Context Switching and could be done automatically using a specialized command object. The only down side is that because SqlCommand is a sealed class we have to use composition rather than inheritance. This may also force us to create a compatible SqlDataAdapter but when all is said and done you have a system that is both scaleable and robust. These changes are also likely to be compatible with SQL Server 2008's CDC technology which can automatically log changes to a table.
-- TSQL TO CREATE A USER WITHOUT A LOGIN
-- AND USE USER CONTEXT SWITCHING
CREATE DATABASE [TestDB]
GO
USE [TestDB]
GO
-- Create the Service User
CREATE LOGIN [ServiceLogin] WITH PASSWORD = 'Uor80$23b91';
CREATE USER [ServiceLogin] FOR LOGIN [ServiceLogin]
GO
-- If we ran this before then we need to drop this user
DROP USER [nzaugg]
GO
-- Create a user without a login
CREATE USER [nzaugg] WITHOUT LOGIN
GO
GRANT IMPERSONATE ON USER::[ServiceLogin] TO [nzaugg]
GO
-- Switch User Context; Optionally Specify 'NO REVERT'
-- If we run this in Query Editor with 'NO REVERT' the
-- only way to go back to our original login is to reconnect!
EXECUTE AS USER = 'nzaugg' --WITH NO REVERT
GO
-- Verify that we are now user 'nzaugg'
SELECT user_name(), suser_name(), original_login()
-- If we used 'WITH NO REVERT' on our EXECUTE AS statment
-- We won't be able to revert and this will throw an exception
REVERT
GO
-- Are we still 'nzaugg'?
SELECT user_name(), suser_name(), original_login()
-- DROP THE DATABASE
DROP LOGIN [ServiceLogin]
GO
USE [master]
GO
DROP DATABASE [TestDB]
GO
Remember, in order to do this all of these users must exist in the database. They must also have rights to perform the operation in the original DML statement. This is where users without logins come in handy (see code lines 18 & 21). The optional WITH NO REVERT will be handy for logging and will further secure our database.
Image 3
EXECUTE AS MSDN Paragraph
SQL Server 2005 Books Online (September 2007)
EXECUTE AS (Transact-SQL)
Sets the execution context of a session.
By default, a session starts when a user logs in and ends when the user logs off. All operations during a session are subject to permission checks against that user. When an EXECUTE AS statement is run, the execution context of the session is switched to the specified login or user name. After the context switch, permissions are checked against the login and user security tokens for that account instead of the person calling the EXECUTE AS statement. In essence, the user or login account is impersonated for the duration of the session or module execution, or the context switch is explicitly reverted. For more information about execution context, see Understanding Execution Context. For more information about context switching, see Understanding Context Switching.
References:
Downloads:
When I seem to have re-occurring themes happen naturally I find that those are good things to blog about. Recently I was speaking with my brother (who has incessant adware problems) about Internet Safety and Cyber Security. A week or two ago I had a stimulating conversation with my friend Anthony about security, and this morning on the Diane Rehm Show there was a segment about Cyber Threats. In which one of the guests stated that he works on Banking systems and will not engage on Online Banking.
The first question anyone has is how wide-spread is this problem and does this really affect me. The answer is that this is wide-spread and it affects everyone whether you own a computer attached to the Internet or not. The scary part is that even the most pragmatic and Internet-savvy users can fall victim. Does this mean you or I should stop using the Internet altogether? Absolutely not. While anyone can fall victim to this kind of threat there are steps to take to greatly reduce your risk!
In this age of technology we have almost no limits to our technical abilities. Unfortunately, our attackers have this same ability.
Types of attacks:
- Trojan Horses - A computer program that poses to be something useful but allows access to your system from the Internet.
- Adware - A computer program that either tracks your usage and sells that information to marketers or pops adds up on your computer.
- Spyware / Key-loggers - A program that "watches" what you do on your computer. They can record every keystroke and send that information to a scammer.
- Worms - A special kind of program (which usually includes spyware or Trojan horses) that spreads its self -- usually through email or mapped network drives.
- Proxy - A term used for an attack coming from a computer in which the user/operator has no idea about. This is a compromised computer system that a remote scammer has installed a Trojan horse on. This "bot" can now do anything it's owner wishes.
- DOS - Denial of Service attack. This is an attack on a server which will render it unable to complete the task it for which it was designed.
- DDOS - Distributed Denial of Service attacks. This type of attack usually involves overwhelming a web site to the point that it can not serve requests to legitimate customers. This usually involves a large amount of "bots" controlled by a single party and often controlled through a mechanism called IRC which is a lot like a chat room.
- Buffer Overflow - this is a special type of attack that targets specific code. Basically if the scammer can pass a malformed piece of data to a function in code then they might craft it in such a way that it will execute part of the data. That allows the attacker to run any kind of code on your machine. Depending on the security of the process that was compromised (which is usually pretty high) they can take over your computer. Remember, any maliciously-crafted data can cause this including data they try to send to you on an unprotected Internet port or data that you requested from a malicious website. Simple things such as an image can contain a buffer overflow attack (and have in the past). This type of attack is not only limited to windows. It can be attributed to careless programming but can often be a weakness in the compiler itself.
- Root Kits - This is a special kind of hacking technique which involves exploiting one small veunerability after another. This is typically on web servers who's upload function is unprotected or ones which have a buffer overflow exploit in place. Once a file is uploaded it is executed and causes a larger hole to be created. Eventually they can take control over that machine.
- Email Scams - Email is where most of the bad stuff originates from. That is because it is cheap and easy to send mail and because it is often easy to harvest or guess an email address. It's far more difficult to get people to visit a malicious website.
Q & A:
- Info: The terms virus's, adware, spyware, and worms can be safely summed by the term malware.
- Q: Are Mac's really more secure than PC's?
- A: Yes and No. Although the Mac has made a comeback the past few years it is still a very, very small percentage of the computers in the world. Because of this most every virus targets a PC running some version of Windows. However, this does not mean that your "safer" using a Mac. As Macs become more more popular more virus will be written to target them and they may have more success than targeting windows. Windows has gone a few rounds of cops and robbers where Macs have not. In my opinion, if you are buying a Mac simply because you think you are "more secure" than don't bother. A sense of false security is the most detrimental risk of all.
- Q: Who is attacking me and why?
- A: Attackers are generally part of 1 of 2 different types. People in it for personal gain and government-sponsored groups. There has been a very significant and organized amount of hacking coming from China which suggests that the Chinese government sponsors this type of activity. Much of their effort seems to be on mapping our resources around the net.
- Q: I get a lot of email about stocks, what is that about?
- A: This is the old pump & dump scam! They artificially inflate the price of "penny stock" that they own a large number of shares of. They send this email en masse telling people to buy lots an lots of this stock. Enough people buy that the stock price raises and the scammer sells the stock and allows it to tank. This kind of scam can be costly for both the business offering the stock and for those foolish enough to actually invest in it.
- Q: Do people really fall for the emails claiming to be from their bank?
- A: No, not really. The problem is that if just one in ten million *do* fall for this scam then it would have been worth it. They can send these phishing emails out at a rate of millions per minuet.
- Q: Will the Internet ever become a safe place.
- A: No. Like the game of cops and robbers this will likely play out forever. The programmer in me wants to believe that it is possible to have 100% secure software. The pragmatist in me knows that it may not ever be possible. However, overall I do tend to believe that it will get much better but will probably get much worse before that begins to happen.
- Q: Does looking at porn on the Internet make me more susceptible to malware?
- A: Absolutely! Porn and Malware go hand-in-hand!
- Q: Does downloading "cracked" programs make me more susceptible to malware?
- A: Absolutely! Crack sites, key sites, etc. are a Trojan horse delivery mechanism. Why do you think these people crack these apps? They do it to lure you there and take control of your computer.
- Q: Can a virus really take control of my email?
- A: Yes, it can but usually it doesn't have to. SMTP (the protocol in which mail is sent over the Internet) has absolutely no good way of verifying that you are who you say you are. If your computer is hacked then it's probably your email address book they are after.
Tips & Tricks:
- Get a home firewall that uses NAT. You may already have this and not realize it but computer systems sitting behind NAT "invisible" to Internet scans which greatly helps keep your computers safe!
- Let your computer update regularly! Make sure auto-update is turned on and working. If the computer needs to restart to apply a patch make sure that happens ASAP.
- Install a software firewall. They will slow down your computer, I know, but they are a necessity today.
- Let your virus scan run weekly
- Own two computers (especially if you have kids). Use one for Internet banking, and Internet purchases, storing personal information and nothing else. Use the other one for everything else, keeping any kind of personal information off of this machine.
- Use the least possible permissions you can for your user accounts.
- Our school used a hardware device in which every time the computer is rebooted the state is restored to exactly the same state every time. This would be a really great tool for your general use computer. (I'll post a link when I can find one)
- If you are at an Internet shopping site and you get a certificate error, leave now! That certificate error is the ONLY thing protecting you from a man-in-the-middle attack!
- NEVER, NEVER, NEVER, NEVER, NEVER download or open an attachment you are not expecting! Even if it looks like it is from a person you trust! If it is from a person you trust verify it's contents before opening! They may have been sent this wonderful screen saver and wanted to share it with you. That's great and all but that screen saver is probably a worm! Also, they may not have actually sent it to you, the screen saver did it!
- Do not download any executable file. Those include files that end with: .exe, .scr, .bat, .pif, .com, .dll, .ocx, .sys Also watch for the space trick where the filename is "myfile.zip .exe". Notice the spaces? You may not see those in Outlook or whatever else you are using.
- Verify from the author any other types of download. Recently virus have been able to attach to innocent PDF files! The moral is that there really is no such thing as an innocent file!
Opinion:
- Virus' are not easy to detect! Virus scanners use something called Heuristics to find virus's, adware, spyware, and worms.
- Both client and server need a way to be independently authenticated by a trusted 3rd party, and if the trust can not be established then there must be no way to continue.
- We need to phase out passwords! They are way too easy to predict and/or capture!
- We need a way to positively identify (for computing purposes) every user on the Internet. This is the only way we can really develop trust relationships with other systems and the only way to end SPAM/Phishing.
Visit these links for more information:
Be careful out there!
Recently, some Linux developers have condemned the practice of shipping drivers without the source in an open petition. The reasoning for such a petition is that "any closed-source Linux kernel module or driver to be harmful and undesirable. We have repeatedly found them to be detrimental to Linux users, businesses and the greater Linux ecosystem."
This is the same kinds of growing pains that Microsoft had in the early days of Windows, esp Windows 95. The Windows OS seemed to be riddled with bugs but the problem was (and always had been) the device drivers and graphics cards are the usual culprit. However, stating simply and closed source drivers are "harmful and undesirable" is a lot like saying that the only to prevent accidents is to not allow anyone to leave their homes. Very few device drivers for windows are open source and there has been tremendous improvement in stability over the past few years. Microsoft has made enormous efforts to train device driver programmers for other companies and has released an extensive DDK and debugging tools. Most recently they even made it so some drivers don't even have to operate in the kernel space but can exist in the user space. This will greatly increase stability of the Operating System by protecting the sensitive kernel. A similar concept could be adopted by the Linux Kernel.
Open source drivers may be a good solution for Linux but this petition makes it sound like it is the only solution for the longevity of Linux. There is a down-side to open access to source code including uncontrolled visioning which can cause worse problems.
--Nathan Zaugg
See also:
http://redmondmag.com/news/article.asp?editorialsid=9992
For now on you may address me as my proper title:
I wish the title was a little less blasphemous; therefore you may alternately call me 'King Nerd'. That would also match with my wife who is a "Slightly Dorky Nerd Queen".
I also scored quite high on the version 1.0 test:
Remember, that is 92 percentile!
--Nathan Zaugg
Those who know me will tell you that Microsoft might as well just put me on payroll because I talk about their new products all day long. I have even characterized myself as an "Unofficial, Unpaid, Microsoft Solutions Evangelist". Having said that, I know what's good and bad about the products I love so much. I also know when someone else has something worth taking a look at.
Google Defined
Google: very successful Internet advertising company seeking to find, buy, and evolve technology that has promise and then figure out a way to turn a profit on the new technology.
And they are very good at what they do. When Google purchased KeyHole (Google Earth) everyone started scratching their heads and wondered why! I may not understand exactly how but I am almost positive that they have turned a profit on it.
Well, here is another interesting gem. Google SketchUp is a armature 3D modeling tool. It's easy to pick up and has some pretty cool features. I created the sketch below of my back yard (or at least how I want it to be). There is tons of detail from the landscape brick in the back to the translucent windows. I did the below model in less than an hour and when I started I couldn't figure out how to add dimension to my square shape or how to move it around.
Cool Features
Some of the cool features are textures, and the 3D model community. The textures makes these simple sketches look very life-like. It has most of the common building materials and many basic outdoor shrubs & plant-life. You can also capture an image from Google Earth and transpose it onto your work area. You can even take a picture of something and mold it to your objects.
I added my model to the Google search and now anyone can use my shop!
The Not So Cool
It would be nice if there was more keyboard involvement. I find my self switching between tools a lot and it's cumbersome. While you can get results pretty quickly the interface is difficult to maneuver even after using it for a while. Simple tasks like changing the size of a rectangle are complicated. You also have an inability (or at least it seems to me for now) to make things exact. I'd like to be able to enter the size of the rectangle and then enter the coordinates. And although it's pretty fast for 3D modeling it does make you wait quite a while for some tasks and I think it could be a whole lot faster! It seems to be written in Ruby and my guess is that it does not take advantage of hardware acceleration and that if it were written in a language like C++ or C# than it would be much faster! Also, I hate the name! Couldnt they have named it Google 3D, G-3D, or even GSketch?
All said, for a free pice of software it does help a lot in convincing my wife that a 30-35 shop will not look too big for our yard!
http://sketchup.google.com/
--Nathan Zaugg
The term "circular dependency" may be foreign to some programmers (especially if you do Java as it is a pretty common practice). However, anyone who has done some scripting for a referential database knows that you have to run scripts in a certain order. Running scripts out of order causes errors when you run. The interesting trick is that if you run that same incorrectly-ordered DDL script again and again you will eventually get it to run without errors. If you were unaware of the order being incorrect and thought to yourself in that moment "Stupid database!" then this blog post is for you!
What is a circular dependency?
It is simply two libraries that use each other (either directly or indirectly) as shown below:
Figure 1: Circular Dependency
Figure 2: Complex Circular Dependency
The complexity of a circular dependency may vary. If you are using Visual Studio and have all of your projects loaded into a single solution AND you add Project References (Right click on project -> Add Reference -> Projects Tab -> {Project Name}) then the IDE will not allow you to create Circular Dependencies. In fact, this is a good practice as Visual Studio will ensure the correct build order.
Why are circular dependencies bad?
Just like our Database example above, a circular dependency makes it so you can not guarantee that your application has the latest code. That is a big deal! Here is why:
- I make changes to Application 1 (in Figure 2)
- I build my project, The changes I made in Application 1 may or may not have gotten into Application 2 (depending on build order). It may have taken a copy of the compiled code that was left over from the last time I built.
- Application 2 depends on this new functionality to provide services to Application 3; This functionality will not work correctly with this build.
- Application 3 may or may not depend on these same services to provide back to Application 1
As you can see in this scenario, there is no such thing as a "correct" build order when there are these circular dependencies. The only way you can arrive at the correct version of the code is to build it as many times as there are nodes in our circle. That would mean for Figure 1 that we would need to build twice and three times for Figure 3. Some of these dependencies can get really ugly! Here is some actual code running in an actual company that I did analysis on some time ago using a tool called Structure 101.
How do I fix circular dependencies?
There are some steps to take to solve even the most complex tangles! They all involve refactoring your code though.
- Refactor common code into a "base" dependency; I usually call this "Common" (figure 3). BEST SOLUTION
- Remove code that is unused. In the tangles shown above many of them are using deprecated/unused code.
- Duplicate the sections of code used. This should be seen as a last resort but given the choice between code duplication and circular dependencies, I take code duplication ever time!
Figure 3: Refactor a Common
Summary
There are two kinds of design concepts for nTier (and other types of architectures as well) called Logical Layout Design and Physical Layout Design. The Logical Layout is simply that your software occupies the same project/package but leverage different classes. In contrast Physical Layout Design forces each tier to be separated into different Projects/Packages. So long as we are careful to manage the dependencies between these packages from the start this is the preferable way to code. While the logical layout does not suffer from the dependency problem eventually you may wish to break these classes apart and find that there are a lot of inner-dependency that should not exist simply because they occupied the same project. Remember to keep it clean!
Microsoft Silverlight 2.0 Beta 2 has been released. This version is supposed to be pretty stable as I understand it. The tools on the other hand, still feel very much like a beta! I had a heck of a time getting this junk installed!
The link to download Silverlight 2.0 Beta 2 for Visual Studio is here: http://silverlight.net/GetStarted/
(note: This installation package includes all you need including Silverlight runtime and SDK)
My Installation Issues
I got to know these dialogs pretty good as I'd tried this many times!
It makes you close all browsers! Ok, sure!
DANG IT! What now?
So you're telling me that I can't have a patch?
Here is the output of the error:
Created new ExePerformer for Exe item
[6/9/2008, 13:48:11] Action: Performing Action on Exe at c:\6472026481f78458c12aa62cd2\silverlight_uninstallrtmpatches.exe...
[6/9/2008, 13:48:11] (IronSpigot::ExeInstallerBase::Launch) Launching CreateProcess with command line = c:\6472026481f78458c12aa62cd2\silverlight_uninstallrtmpatches.exe /q /uninstall
[6/9/2008, 13:49:11] (IronSpigot::ExeInstallerBase::PerformAction) c:\6472026481f78458c12aa62cd2\silverlight_uninstallrtmpatches.exe - Exe installer does not provide a log file name
[6/9/2008, 13:49:11] (IronSpigot::ExeInstallerBase::PerformAction) Exe (c:\6472026481f78458c12aa62cd2\silverlight_uninstallrtmpatches.exe) failed with 0x80070643 - Fatal error during installation. :
[6/9/2008, 13:49:11] (IronSpigot::ExeInstallerBase::PerformAction) PerformOperation on exe returned exit code 1603 (translates to HRESULT = 0x80070643)
[6/9/2008, 13:49:11] Action complete
[6/9/2008, 13:49:11] (IronSpigot::LogUtils::LogFinalResult) Final Result: Installation failed with error code: (0x80070643), Fatal error during installation.
so I hit the net and find these instructions: http://weblogs.asp.net/bradleyb/archive/2008/03/06/installation-tips-for-sivliverlight-tools-beta-1-for-visual-studio-2008.aspx
try to uninstall the KB:
(found under updates)
Kept asking me for some location! Wasn't going to work....what is the backup solution?
Ok, extract the files in the installer...sure.
I put them in a dir called "Install"
Then RUN: msiexec /uninstall VS90-KB949325.msp /L*vx VS90-KB949325-2.log
Double Dang! Ok, whats up??
Hit the web agian: http://silverlight.net/forums/p/17663/58925.aspx
and download MSI ZAP: http://support.microsoft.com/kb/290301
Tripple Dang! Updates are not listed!
Downloaded tool: MSIINV
Download Link:
msiinv.zip
RUN:
msiinv.exe -p > Installed.txt</