Uncategorized – Auto-Magical http://automagical.rationalmind.net n. 1: automatic, but with an element of magic. 2: too complex to understand and/or explain Thu, 07 Apr 2016 05:29:30 +0000 en-US hourly 1 38147107 FEE.org Project Retrospective Part 2: Content Management Systems: Umbraco, WordPress, or Custom? http://automagical.rationalmind.net/2015/09/10/fee-org-project-retrospective-part-2-content-management-systems-umbraco-wordpress-custom/ http://automagical.rationalmind.net/2015/09/10/fee-org-project-retrospective-part-2-content-management-systems-umbraco-wordpress-custom/#respond Thu, 10 Sep 2015 16:53:41 +0000 http://automagical.rationalmind.net/?p=932 Continue reading FEE.org Project Retrospective Part 2: Content Management Systems: Umbraco, WordPress, or Custom? ]]> In part 1 of this series, I provided 6 time management tips for independent developers.  In this post, I will discuss why I chose Umbraco as a CMS for this project.  Part 3 will cover best-practices and tools for managing self-managed projects – people management, task management, quality assurance, deployments, sales, and communications.  Part 4 will cover content migration, custom feature implementation, launch planning, performance optimization, Amazon Web Services integration, and team engagement

WordPress, Umbraco, or Custom?

Background:

Most web developers who were around before the explosion of modern content management applications have built one themselves. I built my first CMS using Classic ASP and Access around 2002 for a university newspaper. In 2004, I designed a new CMS for Mises.org and evolved it through the latest Microsoft technologies for just over 10 years.  (More on this project here.) It was a great learning experience to build the site from a simple article archive to a million+ visits per month. During that time, I build sites using Joomla, WordPress, and Umbraco, so I was able to appreciate the costs and benefits of build a CMS from scratch.

Running a custom CMS was fun and educational, but there were many downsides. Building a popular website is a huge investment, and requires specialized expertise. It requires extensive knowledge about search engine optimization, content workflows, user experience design, full-stack performance optimization and much more. Software development in general is difficult, risky and requires continuing effort to stay up to date.

Build custom software is fun, but the criteria in choosing a CMS strategy should be to minimize technical investment and leverage third party experience as much as possible.

Selecting a CMS:

Here are the options I considered:

  1. A commercial, proprietary application with most required features built in such as SiteFininity, SiteCore, ExpressionEngine, etc
  2. An open-source (free) application such as WordPress, Drupal, and Umbraco.
  3. A custom CMS, written from scratch

Benefits of a commercial CMS:

  • Well supported by vendor.
  • Has most needed features out of the box.

Disadvantages:

  • Can be expensive. The median Adobe CQ/AEM licensing cost is $450K and median install cost is $2 million. (I had an opportunity to work an on Adobe AEM project – it’s a great tool, but I find it hard to recommend the $2 million premium over the free alternatives for most organizations.)
  • More likely to undergo major changes in future releases such as doubling in cost, targeting a different market, discontinuing product, etc.
  • No access to source code means core changes to functionality may be impossible.
  • Usually have a smaller development community and fewer and more expensive add-ons.

Benefits of an open-source CMS:

  • For popular apps, have millions of users, with community support
  • More likely to have add-ons which provide needed features
  • Access to source code means anything is customizable
  • Usually code base and features stable over long periods of time and upgrades are possible

Disadvantages:

  • Provide a set of different features often requires cobbling together plugins by different authors, with very different quality
  • The above can lead to poor performance and poor integration
  • Lack of vendor support, or perhaps expensive support by a third party

Benefits of a custom CMS:

  • Very fast and storage-efficient database and code are possible by avoiding layers of abstraction.
  • No limits on customization
  • Potentially more secure and spam proof, but avoiding bots and known exploits of popular applications
  • A completely proprietary implementation provides exactly the solution that a client needs – assuming (!) that they know what they need and it does not change over time.

Disadvantages:

  • Long-term cost of implementing features which already exist in a third party CMS

For the project of implementing a new CMS for fee.org, I considered the following candidates:

  1. WordPress: the most popular CMS with 50% of market
  2. Umbraco: an open-source .Net CMS with only about .1% of market. It’s high quality, mature and well-supported by the community
  3. Sitefinity 1% share – Commercial .Net CMS
  4. SiteCore .3% share Commercial .Net CMS
  5. ExpressionEngine $299+support PHP CMS/MySQL. 1.5%
  6. Custom CMS: write the FEE.org CMS from scratch

Top two: WordPress vs Umbraco

Ultimately, it came down to WordPress vs Umbraco:

In regard to WordPress, building Liberty.me exposed me to the pros and cons of WordPress.  I decided to use WordPress for Liberty.me because we needed to get a massive amount of functionality implemented quickly, and WordPress was the only platform with development ecosystem to make that possible. It was easy to source talent and find a plugin that did everything we needed, but we ended up with a bloated site (over 120 plugins) that stepped all over each other and required massive optimization and many layers of caching to perform well. We use nginx, varnish, hhvm, redis, three specialized CDN providers, Amazon AuroraDB and much more to get the site to perform acceptably.

Umbraco was a natural choice because of my successful experience building Hersheys Kitchens using Umbraco 4.0, and the great features it had added since then.

Winner: Umbraco

  • It is a blank slate with a lot of flexibility to create great customer experiences.
  • It is a fast and stable ASP.Net platform (with queries using the on Lucene engine) that works well out of the box.
  • It is open source, but commercially supported Great free and paid components are available
  • I’m an expert .Net/Microsoft developer, but junior in PHP, so I would have to rely on contractors much more if we went with WordPress.

Stay tuned for details of how I implemented this project in the next post…

 

]]>
http://automagical.rationalmind.net/2015/09/10/fee-org-project-retrospective-part-2-content-management-systems-umbraco-wordpress-custom/feed/ 0 932
Bitcoin Exchange Project Part 2: Order Matching Algorithm http://automagical.rationalmind.net/2013/10/16/bitcoin-order-matching-algorithm/ http://automagical.rationalmind.net/2013/10/16/bitcoin-order-matching-algorithm/#comments Wed, 16 Oct 2013 08:44:32 +0000 http://automagical.rationalmind.net/?p=888 Continue reading Bitcoin Exchange Project Part 2: Order Matching Algorithm ]]> Introduction to this project.

Order Matching Algorithm Description (Rough Draft – 2013/10/16)

Summary: A currency exchange is a system for buyers and sellers of different currencies to exchange different types of currency. The other matching module matches buy and sell orders, creates transactions to record the process, and updates the customers account balances.

Part One: Basic Order Processing

Action: A customer enters the quantity and prices of the orders and clicks “buy” or “sell”

1: Website creates record in OrderBook with Pending order status. The order is filed to be processed. [PlaceBuyBid

Then, the Order Matching Service iterates through the list of pending orders. [public int PlaceBuyBid(int customerId, decimal quantityOfBTC, decimal pricePerBTC, DateTime ?expirationDate = null) and public int PlaceSellOffer(int customerId, decimal quantityOfBTC, decimal pricePerBTC, DateTime ?expirationDate = null)] For each order:

2: Re-verify order status to ensure that it is still pending/active. If expired, the order is Cancelled

3: Validate the order funding. [public bool ValidateOrderFunding(Order order)] The customer must have sufficient assets to process the order. If not, the order is Suspended (it may be re-actived if funds become available later.) If order passes validation:
a: the order status is changed to Active
b: the assets needed to pay for the order are added to the Frozen balance. The prevents the customer from placing orders on more assets than he has. This feature may be removed later – we can allow spending greater than the available balance if we check the balance before processin the transaction.

4: The Order Matching Service tries to find a match to the buy or sell order. To find a match, we search all Active orders which match the specified price. [ISpecification IsMatchingOrderQuery(decimal price, int orderTypeId, int wantAssetTypeId,int offerAssetTypeId, bool? isMarketOrder)]

*If the order is a buy, we look for a price less than or equal.
*If the order is a sell, we look for a price greater than or equal.
*If it’s a market order, we find the highest (sell) or lowest (buy) price.

We sort the matches ascending for buy orders, and descending for sell orders. Then we sort by date if the price matches.  [ISpecification IsMatchingOrderQuery(decimal price, int orderTypeId, int wantAssetTypeId,int offerAssetTypeId, bool? isMarketOrder)]

5: We load the top 3 matches into memory. The reason we take 3 matches is in case one of the orders fails validation, we can try with the order 2 orders.

6: We compare the order and the match. This is a double check with C# – the order should have been already matched by the database query above. The orders should match: assets ($/BTC, order types (buy/sell),non-two market orders, and have matching prices (as above). [OrderComparisonResult CompareOrders(Order firstOrder, Order secondOrder)]

7: If the orders comparison succeds, we generate a Transaction to record the match [Transaction GetTransactionForTwoOrders(OrderComparisonResult comparisonResult)].
*A_Order is the buy order
*B_Order is the sell order
(A and B are used because I’m still not sure whether ordering should be Buy/Sell, chronological, or something else)

8: If the order quantities do not match exactly, we must generate a split order with the remainder of the larger order.

9: We run [ActivateTakeProfitAndStopLossOrders(Order order)] – TODO – should not be done here, but scheduled. (See separate post for advanced orders – take proft, stop loss, trigger, stop order, etc.)

10: We process the transaction to record the result [public Order ProcessTransaction(Transaction transaction)] (note: this module is a database transaction)

a: add the transaction and the split orders to the DB
b: for both orders in the transaction:

  •  subtract the debit access (customers balance of $ or BTC)
  •  increment the credit asset (customers balance of $ or BTC)
  •  record the commision in the commision account
  •  unfreeze frozen balances
  •  save changes

11: Repeat the process on each split order, until there is either no remaining quantity (entire order is fullfilled) or we
run out of matching orders; the remaining split order stays open as an active order


foreach (OrderProcessResultModel n in ProcessOrder(splitOrder.OrderId))
{
yield return n;
}

12: If the remaining quantity is 0, set the status to Completed

OrderBook Schema

]]>
http://automagical.rationalmind.net/2013/10/16/bitcoin-order-matching-algorithm/feed/ 6 888
New app: CryptoPad, a simple AES client http://automagical.rationalmind.net/2013/07/06/app-cryptopad-simple-aes-client/ http://automagical.rationalmind.net/2013/07/06/app-cryptopad-simple-aes-client/#respond Sat, 06 Jul 2013 12:52:57 +0000 http://automagical.freecapitalists.org/?p=875 CryptoPad is a simple AES256 encryption/decryption app.

Download the executable or compile it from source: GitHub

Let me know if you’d like a version for OS X, etc.

cryptopad
]]>
http://automagical.rationalmind.net/2013/07/06/app-cryptopad-simple-aes-client/feed/ 0 875
Creating a .Net compatible code signing certificate http://automagical.rationalmind.net/2013/06/14/creating-net-compatible-code-signing-certificate/ http://automagical.rationalmind.net/2013/06/14/creating-net-compatible-code-signing-certificate/#respond Fri, 14 Jun 2013 10:56:17 +0000 http://automagical.freecapitalists.org/?p=867 Continue reading Creating a .Net compatible code signing certificate ]]> The purpose of this workaround is to bypass the lack of support for CNG certificates in the .Net Framework.   I used OpenSSL to convert the certificate obtained from the Certificate Authority (Verisign, Thawte, etc) to a format supported by .Net.  This tutorial builds on the workaround in the Microsoft Connect bug report.

Ingredients:

1: Certificate text

2: Certificate imported into Windows certificate store from a root CA

3: OpenSSL

Steps:

1: Create a new file (CERT_ONLY.crt) with the certificate (text from —-BEGIN CERTIFICATE—– to —–END CERTIFICATE—–)

2: Import the certificate into certificate store via the CA website, then export it to file EF.pfx.  Include the private key.

3: Generate PEM file with the private key only:

openssl pkcs12 -in EF.pfx -nocerts -out EF.pem

4: Convert private key to RSA format

openssl rsa -in EF.pem -out EF_RSA.pem

5: Generate the code signing certificate

 openssl pkcs12 -export -out EFnew.pfx -inkey EF_RSA.pem -in CERT_ONLY.crt

6: Delete the existing certificate from the certificate store (backup first) then import the newly generated certificate

]]>
http://automagical.rationalmind.net/2013/06/14/creating-net-compatible-code-signing-certificate/feed/ 0 867
Amazon S3 Upload Reliability Test Tool http://automagical.rationalmind.net/2013/04/16/amazon-s3-upload-reliability-test-tool/ http://automagical.rationalmind.net/2013/04/16/amazon-s3-upload-reliability-test-tool/#respond Tue, 16 Apr 2013 11:21:34 +0000 http://automagical.freecapitalists.org/?p=858 An app created to stress-test the speed and reliability of uploads to Amazon S3.  (It deletes the uploaded files in a background thread.)

You can select 1-10,000 uploads of the same file.   Download the source here  or from GitHub 

TestTool

 

]]>
http://automagical.rationalmind.net/2013/04/16/amazon-s3-upload-reliability-test-tool/feed/ 0 858
Converting Picasa ini dates to directory dates http://automagical.rationalmind.net/2013/03/05/converting-picasa-ini-dates-directory-dates/ http://automagical.rationalmind.net/2013/03/05/converting-picasa-ini-dates-directory-dates/#respond Tue, 05 Mar 2013 13:12:10 +0000 http://automagical.freecapitalists.org/?p=851 Continue reading Converting Picasa ini dates to directory dates ]]> One issue I faced in the process of migrating from Picasa to Adobe Lightroom is that Picasa stores event dates in .picasa.ini files. I had spent many hours setting correct dates in my events (folders) and wanted to import them. So I wrote the following C# script (should work for Windows and OS X – I ran in via Mono).

Update: This code is on github @ https://github.com/DavidVeksler/EmbedPicasaProperties

The below will not be updated.

using System;
using System.IO;
 
namespace ConvertPicasaToFolderDates
{
	class MainClass
	{
		private const string IniFileName = @".picasa.ini";
 
		private static void Main(string[] args)
		{
			string startFolder = @"/Users/davidv/Pictures/";
 
			Console.WriteLine(startFolder);
 
			UpdateFoldersInPath(startFolder);
 
		}
 
		private static void UpdateFoldersInPath(string folder)
		{
			string[] directories = Directory.GetDirectories(folder);
 
			if (directories.Length > 0)
				Console.WriteLine("{0} folders in {1}", directories.Length, folder);
 
			foreach (string directory in directories)
			{
				try
				{
					string iniPath = Path.Combine(directory, IniFileName);
 
					if (File.Exists(iniPath))
					{
						Console.WriteLine("Parse " + iniPath);
 
						foreach (string line in File.ReadAllLines(iniPath))
						{
							// date=29586.603900
							if (!line.StartsWith("date=")) continue;
 
							string dateString = line.Substring(5);
 
							DateTime date = ConverPicasaDateToDateTime(dateString);
 
							DateTime originalTime = Directory.GetCreationTime(directory);
 
							if (originalTime != date && date.Year < 2010)
							{
								Console.WriteLine("{0} to {1}", originalTime, date);
								Directory.SetCreationTime(directory, date);
// (For Mono SetCreationTime did not work correctly, so use SetLastWriteTime.)
								Directory.SetLastWriteTime(directory,date);
							}
 
							break;
						}
					}
				}
				catch (Exception ex)
				{
					Console.WriteLine(ex);
				}
 
				try
				{
					UpdateFoldersInPath(directory);
				}
				catch (Exception ex)
				{
					Console.WriteLine(ex);
				}
			}
		}
 
		// convert =29586.603900 to date time format
		private static DateTime ConverPicasaDateToDateTime(string dateString)
		{
			var startDate = new DateTime(1900, 1, 1);
 
			DateTime date = startDate.AddDays(Convert.ToDouble(dateString) -2);
 
			Console.WriteLine("new date: " + date);
 
			return date;
		}
	}
}

 

]]>
http://automagical.rationalmind.net/2013/03/05/converting-picasa-ini-dates-directory-dates/feed/ 0 851
“It’s a Girl!” OS X Mail Stationery Template http://automagical.rationalmind.net/2013/02/21/girl-baby-mail-stationery-template/ http://automagical.rationalmind.net/2013/02/21/girl-baby-mail-stationery-template/#comments Thu, 21 Feb 2013 17:58:35 +0000 http://automagical.freecapitalists.org/?p=844 Screen Shot 2013-02-22 at 1.53.26 AM

I’m not a great designer, but here is a girl version of the built in Baby mail stationery template for OS X Mail. It’s pink-ish all over.

Unzip this file into /Library/Application Support/Apple/Mail/Stationery/Apple/Contents/Resources/Announcements/Contents/Resources

]]>
http://automagical.rationalmind.net/2013/02/21/girl-baby-mail-stationery-template/feed/ 4 844
OS X + Eclipse + Mono Tutorial http://automagical.rationalmind.net/2008/12/31/os-x-eclipse-mono-tutorial/ http://automagical.rationalmind.net/2008/12/31/os-x-eclipse-mono-tutorial/#respond Wed, 31 Dec 2008 17:25:23 +0000 http://dotmac.rationalmind.net/2008/12/os-x-eclipse-mono-tutorial/ Continue reading OS X + Eclipse + Mono Tutorial ]]> Here is an interesting tutorial for developing .Net apps in OS X using the Eclipse IDE. The XCode equivalent is CSharpPlugin. If a native IDE isn’t important to you, running Visual Studio via VMware works just fine too.

Update: from IBM: Migrate .NET applications from Visual Studio to Eclipse: The Emonic plug-in and NAnt give .NET and C# capability to Eclipse

]]>
http://automagical.rationalmind.net/2008/12/31/os-x-eclipse-mono-tutorial/feed/ 0 89
Why no Paint? http://automagical.rationalmind.net/2008/12/16/why-no-paint/ http://automagical.rationalmind.net/2008/12/16/why-no-paint/#respond Tue, 16 Dec 2008 07:04:08 +0000 http://dotmac.rationalmind.net/?p=75 Continue reading Why no Paint? ]]> macpaintWhy is there no basic paint application in OS X? Sure, there are dozens of free applications that do the same thing, but it seems like a useful thing to have for users who don’t need anything more complex.  After all, a drawing application was the first application created for the Macintosh, before even the operating system.  I’m sure there is a philosophical or marketing reason behind it, but I can’t think of any.

Most adult Windows users who use Microsoft’s Paint actually use it to save screen shots, which is easier in OS X with Preview’s “Grab” menu or keyboard shortcuts.

]]>
http://automagical.rationalmind.net/2008/12/16/why-no-paint/feed/ 0 75
Using the new ClearType fonts on your Mac http://automagical.rationalmind.net/2008/11/21/using-the-new-cleartype-fonts-on-your-mac/ http://automagical.rationalmind.net/2008/11/21/using-the-new-cleartype-fonts-on-your-mac/#respond Fri, 21 Nov 2008 19:01:18 +0000 http://dotmac.rationalmind.net/?p=34 Continue reading Using the new ClearType fonts on your Mac ]]> One of the nice touches of OS X is the use of the Helvetica typeface for rich-text editing in TextEdit and other programs. Microsoft’s version of Helvetica is Arial, which is basically a lower-quality rip-off of the original. (To understand my attraction to Helvetica, I recommend watching Helvetica, the documentary.)

However, in Vista/Office 2007, Microsoft released the ClearType Font Collection, a great new set of fonts, including several designed especially for viewing text on computer screens. The two I find particularly useful in this regard are Cambria, which is optimized for viewing small text, and Consolas, which is a monospaced font useful in programming and the like.

So how do you get these fonts in your Mac? Well, they come with Office 2008 for the Mac. After you install office, just go into your favorite editor’s preferences and select Cambria and/or Consolas as your default font.

If you don’t have Office 2008, things are a little trickier. This tutorial will guide you through installing the fonts in Windows and Linux – and OS X, if you install the required Linux utilities via something like MacPorts. Office 2004 users can get some of the fonts with the Open XML Converter. If you have a Vista computer, you can copy them from C:WINDOWSFonts to /Library/Fonts. If you have a pre-Vista OS, you can get them with the free Powerpoint 2007 viewer. If you are still out of luck, you can always purchase them directly from the foundry.

]]>
http://automagical.rationalmind.net/2008/11/21/using-the-new-cleartype-fonts-on-your-mac/feed/ 0 34