This blog is where I post about my consulting work with Microsoft Technologies, and other random tidbits that don't fit in my Photo Blog or my Iraq Blog.

Monday, November 9, 2009

ScrappyDB: C# Object Mapper for Amazon SimpleDB

I've been working on a new C# Object Mapper for Amazon SimpleDB for a few months to provide an "infinitely" scalable back end for a personal web project, and today I'm releasing it on CodePlex.


Project Description

ScrappyDB is a simple object mapper for storing .NET classes and collections in Amazon SimpleDB. It is currently under active development to support the needs of a specific website. As such it only provides functionality to support the needs of one "customer", this means that there are many "obvious" use cases and data types that are not yet supported... because we didn't need them. But we think other people may find it useful, so we are sharing it here...

Why create an new tool instead of using one of the existing .NET tools for SimpleDB?

In particular:
  • Simple Savant http://simplesavant.codeplex.com/ (Simple Savant is a great tool and definitly a more mature code base. We would recommend that you investigate it first before considering ScrappyDB.)

Features that we felt were important for our implementation:
  • Support for multi-value attributes (using arrays or arraylists).
  • Support for nested objects and relationships (in particular cases where a custom class has a property that is a collection of some other custom class).
  • Integrated support for web caching for performance.
  • Location data (in particular the ability to do proximity searches "show me all records within 5 miles of X").

Things that we did NOT feel were important for our implementation:
  • Support for all .NET data types (we only support a subset of "common" data types, but there is no reason you can't enhance it to support the one you need).
  • Update caching to work around "eventual consistency" (if you don't have a full understanding of what eventual consistency means take the time to learn about it before you decide whether SimpleDB is a viable platform for your data).

About the source code
  • To do what it does ScrappyDB makes extensive use of some painful reflection.
  • The code does not follow any identifiable design patterns and it doesn't follow any of the latest inversion of control fashions: "I'm just a caveman, and I'm not familiar with your modern ways..."

Known Issues:
  • Current code is definitely ALPHA, has not had any significant production use.
  • Relationship logic is lightly testing, is probably buggy and is likely to see some significant refactoring.
    • We are planning to add a second relationship type that uses multivalued attributes for "1 to few" relationships ("1 to many" relationships use an intermediate relationship "table" which is slower).

  • Currently the primary developer documentation for ScrappyDB is the unit tests included with the source code.

But here are a few basic examples to help you get started:

Basic Save
            var db = new DB();
var o = new MyClass {Id = Guid.NewGuid, TestProperty = "HelloWorld"};

Basic Load by Id
            var result = db.Load(testId);

Save a Collection
            var a = new ScrappyDbCollection();
a.Add(new TestItem() { Id = Guid.NewGuid(), TestString = "test1" });
a.Add(new TestItem() { Id = Guid.NewGuid(), TestString = "test2" });
a.Add(new TestItem() { Id = Guid.NewGuid(), TestString = "test3" });


Query a Collection
            var b = new ScrappyDbCollection();
b.QueryCollection(" TestString = 'test2' ");

No comments: