Bret Koppel's Blog

Project Euler Problem 3

2013-08-30T00:00:00-04:00

Lately I’ve been exploring Project Euler as a way to get started with Clojure. This post will feature solutions to Project Euler Problem 3 in C# and Clojure, so if you’re trying to avoid spoilers then now’s your chance to look at kittens instead.

Problem 3 is defined as:

The prime factors of 13195 are 5, 7, 13 and 29.

What is the largest prime factor of the number 600851475143 ?

You can brute force this pretty easily using a pre-generated prime list. If you know nothing else about prime numbers, you can reason that since 2 and 3 are both prime, then given an odd number n the largest factor must be $\leq{n \over 3}$ . Using our list of primes as a sanity check, we can run the following and then try our matches until Project Euler lets us through.

var primes = new[] {2,3,5,7,11,13,17,19,23,29,31,37,41,43,47,53,59,61,67,71,73,79,83,89,97,101,103,107,109,113,127,131,137,139,149,151,157,163,167,173,179,181,191,193,197,199,211,223,227,229,233,239,241,251,257,263,269,271,277,281,283,293,307,311,313,317,331,337,347,349,353,359,367,373,379,383,389,397,401,409,419,421,431,433,439,443,449,457,461,463,467,479,487,491,499,503,509,521,523,541,547,557,563,569,571,577,587,593,599,601,607,613,617,619,631,641,643,647,653,659,661,673,677,683,691,701,709,719,727,733,739,743,751,757,761,769,773,787,797,809,811,821,823,827,829,839,853,857,859,863,877,881,883,887,907,911,919,929,937,941,947,953,967,971,977,983,991,997,1009};
const long limit = 600851475143;
var ceiling = (long) Math.Ceiling(limit/3.0);

for (long tester = 3; tester < ceiling; tester+=2)
{
  if (limit%tester == 0 && primes.All(m => tester % m > 0))
    Console.WriteLine("Prime Candidate: " + tester);
}

If you’re planning to run this, I’d also plan on taking a break from the computer. A long break. Crunching this took 25 minutes on my setup and, frankly, the pre-generated prime list feels a little like cheating. Now the fun can begin. What can we do to speed this up?

If we had a primality test then we could use that instead of relying on our pre-generated list of primes. We could also verify that the number isn’t prime before going through all of our calculations.
If we know a number n isn’t prime, we can use simple trial division and limit our search to numbers less than $\sqrt n$ .

C# doesn’t include a built in primality test, so our first task is to build that out(or grab an existing implementation ). I chose to build out the Miller-Rabin test:

public static bool IsPrimeByMillerRabin(BigInteger candidate, int numberOfRounds = 5)
{
  if (candidate <= 3)
    return true;

  if (candidate % 2 == 0)
    return false;

  var s = 0;
  var d = candidate - 1;
  while (d % 2 == 0)
  {
    s++;
    d = d / 2;
  }
  
  var rand = new Random();
  for (int i = 0; i < numberOfRounds; i++)
  {
    var a = rand.Next(2, candidate > int.MaxValue ? int.MaxValue - 2 : (int)candidate - 2);
    var c = BigInteger.ModPow(a, d, candidate);
    if (c == 1 || c == candidate-1)
      continue;

    for (int r = 0; r < s; r++)
    {
      c = BigInteger.ModPow(c, 2, candidate);
      if (c == 1)
        return false;

      if (c == candidate - 1)
        break;

      if(r == s-1)
        return false;
    }
  }

  return true;
}

With our prime test in place, we can move on to implementing trial division:

public static void Problem3_TrialDivision()
{
  Console.WriteLine(PrimeFactorsByTrialDivision(600851475143).Max());
}

public IEnumerable<BigInteger> PrimeFactorsByTrialDivision(BigInteger value)
{
  if (IsPrimeByMillerRabin(value, 5))
    return new[] {value};

  var primeFactors = new HashSet<BigInteger>();
  while (value % 2 == 0)
  {
    primeFactors.Add(2);
    value /= 2;
  }

  var ceiling = new BigInteger(MathHelpers.Sqrt(value)) + 1;
  for (BigInteger tester = 3; tester < ceiling; tester += 2)
  {
    if (value % tester == 0 && IsPrimeByMillerRabin(tester, 5))
    {
      value /= tester;
      primeFactors.Add(tester);
      if (IsPrimeByMillerRabin(value))
      {
          primeFactors.Add(value);
          break;
      }

      tester -= 2;
    }
  }

  return primeFactors;
}

This quickly yields our answer without any of the nasty hacks we had in the first(very slow) solution. Before we go any further, let’s see what this might look like in Clojure. Java provides a convenient isProbablePrime method so we can do the following:

(defn prime-factors-by-trial-division
  "Gets the frequencies of each prime factor for value."
  [value] 
  (loop [v value testers (range 3 value 2) primes []]
    (let [t (first testers)]
      (cond
        (< v 3) (frequencies (conj primes v))
        (.isProbablePrime (BigInteger. (str v)) 3) (frequencies (conj primes v))
        (zero? (mod v 2)) (recur (/ v 2) testers (conj primes 2))
        (and (.isProbablePrime (BigInteger. (str t)) 3) (zero? (mod v t))) (recur (/ v t) testers (conj primes t))
        :else (recur v (next testers) primes)
      )
    )
  )
)
(println "Max Factor: " (first (sort-by - (keys (prime-factors-by-trial-division 600851475143)))))

Trial division on a number of this size is plenty fast, but I wanted to go a little further down the rabbit hole. After all, looking ahead at the Project Euler problem list makes it seem like some additional prime-related methods might come in handy. Looking up the pages on integer factorization and prime numbers introduces the idea of sieves. Would it be faster to generate all primes up to a number and then test only those values? I took a stab at recreating the sieve of Atkin to find out.

public static ICollection<BigInteger> PrimesByAtkin(BigInteger limit)
{
  var primes = new HashSet<BigInteger> { 2, 3, 5 };
  if (limit <= 5)
    return primes.Where(m => m <= limit).ToArray();

  var cachedPrimes = MemoryCache.Default.Get("LargestAtkin");
  if (cachedPrimes != null)
  {
    var castPrimes = (KeyValuePair<BigInteger, ICollection<BigInteger>>)cachedPrimes;
    if (limit == castPrimes.Key)
      return castPrimes.Value.ToArray();
    
    if (limit < castPrimes.Key)
      return castPrimes.Value.Where(m => m <= limit).ToArray();
  }

  var flip1 = new BigInteger[] { 1, 5 };
  var flip2 = new BigInteger[] { 7 };
  var flip3 = new BigInteger[] { 11 };

  // no built in sqrt, but MSDN offers this(http://msdn.microsoft.com/en-us/library/dd268263.aspx)
  var sqrt = new BigInteger(MathHelpers.Sqrt(limit));
  var numbers = new HashSet<BigInteger>();
  Action<BigInteger> flipNumbers = (n) =>
  {
    if (!numbers.Add(n))
      numbers.Remove(n);
  };

  for (BigInteger x = 1; x <= sqrt; x++)
  {
    for (BigInteger y = 1; y <= sqrt; y++)
    {
      BigInteger n = (4 * (x * x)) + (y * y);
      if (n <= limit && (flip1.Contains(n % 12)))
        flipNumbers(n);

      n = (3 * (x * x)) + (y * y);
      if (n <= limit && (flip2.Contains(n % 12)))
        flipNumbers(n);

      n = (3 * (x * x)) - (y * y);
      if (x > y && n <= limit && (flip3.Contains(n % 12)))
        flipNumbers(n);
    }
  }

  var sortedNumbers = numbers.OrderBy(m => m);
  foreach (var sieveNumber in sortedNumbers)
  {
    if (!numbers.Contains(sieveNumber))
      continue;

    primes.Add(sieveNumber);
    var squareSieve = sieveNumber * sieveNumber;
    var multiplier = 1;
    while (multiplier * squareSieve <= limit)
    {
      numbers.Remove(multiplier * squareSieve);
      multiplier += 1;
    }
  }

  MemoryCache.Default["LargestAtkin"] = new KeyValuePair<BigInteger, ICollection<BigInteger>>(limit, primes);
  return primes;
}

public IEnumerable<BigInteger> PrimeFactorsByAtkin(BigInteger value)
{
  // verify the number isn't prime
  if (IsPrimeByMillerRabin(value, 10))
    return new[] { value };

  var firstPrimes = PrimesByAtkin(100).OrderBy(m => m);
  var primes = firstPrimes.Where(m => value % m == 0).ToList();
  var primeFactors = new HashSet<BigInteger>(primes);
  do
  {
    value = primes.Aggregate(value, (current, prime) => current/prime);

    if (IsPrimeByMillerRabin(value, 5))
    {
        primeFactors.Add(value);
        return primeFactors;
    }
    primes = firstPrimes.Where(m => value % m == 0).ToList();
  } while (primes.Any());

  return primeFactors.Union(PrimeFactorsByTrialDivision(value));
}

This was about 50% faster than the previous version and there’s still room for improvement. I might post a follow-up about some of the other solutions I’ve implemented - including one that uses a rational sieve - but this seems like plenty for now. Let me know about your great solutions(or bugs in my code) by contacting me on Twitter.

Project Euler Problem 3 was originally published by Bret Koppel at Bret Koppel's Blog on August 30, 2013.

Project Euler Problem 2

2013-08-24T00:00:00-04:00

Lately I’ve been exploring Project Euler as a way to get started with Clojure. This post will feature solutions to Project Euler Problem 2 in C# and Clojure, so if you’re trying to avoid spoilers then now’s your chance to look at puppies instead. Problem 2 is defined as:

Each new term in the Fibonacci sequence is generated by adding the previous two terms. By starting with 1 and 2, the first 10 terms will be:

1, 2, 3, 5, 8, 13, 21, 34, 55, 89, …

By considering the terms in the Fibonacci sequence whose values do not exceed four million, find the sum of the even-valued terms.

I like to start out with the simplest approach I can find. A plain old loop should do the trick:

public static void Problem2_Loop()
{
  var sum = 0;
  var x = 0;
  var y = 1;
  int nextFib;
  do
  {
      nextFib = x + y;
      if (nextFib % 2 == 0)
          sum += nextFib;

      x = y;
      y = nextFib;
  } while (nextFib < 4000000);

  Console.WriteLine(sum);
}

This seems pretty efficient. I wanted to give recursion a shot, though, just for comparison’s sake:

public static void Problem2_Recursive()
{
  Console.WriteLine(EvenFibonacciToLimit(4000000).Sum());
}

private static IEnumerable<int> EvenFibonacciToLimit(int limit)
{
    var tester = new Func<int, bool>((item) => item % 2 == 0);
    return FibonacciToLimit(limit, null, tester);
}

private static IEnumerable<int> FibonacciToLimit(int limit, List<int> sequence = null, Func<int, bool> tester = null, List<int> filteredSequence = null)
{
    if (sequence == null)
        sequence = new List<int> { 0, 1 };

    var nextFib = sequence[sequence.Count - 1] + sequence[sequence.Count - 2];
    if (nextFib >= limit)
        return tester == null ? sequence : filteredSequence;

    sequence.Add(nextFib);
    if (tester != null && tester(nextFib))
    {
        if (filteredSequence == null)
            filteredSequence = new List<int>();

        filteredSequence.Add(nextFib);
    }

    return FibonacciToLimit(limit, sequence, tester, filteredSequence);
}

How do the two stack up? The loop is faster, averaging <1ms per 1000 calls, while the recursive solution takes ~1ms. This makes sense given that the recursive solution has the List overhead. We could strip those out in favor of ints to speed it up, as I do in my Clojure solution. As always, you can plug this into the online REPL to verify it:

(defn fib-even [limit]
  (loop [x 0 y 1 acc 0]
   (let [term (+ x y)]
    (if (< y limit)
     (recur y term (if (even? term) (+ acc term) acc))
     acc))))

(println (fib-even 4000000))

This averages 1.2μs per call and seems pretty clean to me. Note that the benchmarks aren’t comparable between languages as they’re running on totally different machine configurations. Do you have a better solution? Let me know on Twitter!

Project Euler Problem 2 was originally published by Bret Koppel at Bret Koppel's Blog on August 24, 2013.

Project Euler Problem 1

2013-08-21T00:00:00-04:00

Lately I’ve been exploring Project Euler as a way to get started with Clojure. This post will feature solutions to Project Euler Problem 1 in C# and Clojure, so if you’re trying to avoid spoilers then now’s your chance to look at kittens instead. Problem 1 is defined as:

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.

Find the sum of all the multiples of 3 or 5 below 1000.

What I love about the problems is that each one is as hard as you want to make it. Problem 1, for instance, is doable as a one-liner in C#:

// Slow
// Why doesn't range just take actual boundaries? The world may never know.
Console.WriteLine(
  Enumerable.Range(3, 997)
  .Where(m => m % 3 == 0 || m % 5 == 0)
  .Sum()
);

Easy, right? I wonder if a swap to Aggregate will speed it up at all.

// Faster
Console.WriteLine(
  Enumerable.Range(3, 997)
  .Aggregate(0, (acc, item) => acc + ((item % 3 == 0 || item % 5 == 0) ? item : 0))
);

Neither of these really appeal to me, though. Why generate 999 elements when we’re only ever going to use a few of ‘em?

// Fastest
var sum = 0;
var multiplier = 1;
while (multiplier*3 < 1000)
{
  sum += multiplier*3;
  var five = multiplier*5;
  if (five < 1000 && five%3 != 0)
    sum += five;

  multiplier += 1;
}

Alright, let’s see how these compare. Here’s how the benchmarks work out over 1000 runs in the VM I’m using:

Aggregate provides a small advantage over filtering and summing, but neither comes close to just calculating what we need. Not a bad speed increase if you’re willing to write a few extra lines of code, right? Let’s move on to Clojure. Having started looking at Clojure only a few days ago, my solutions aren’t going to be idiomatic or super efficient and I’m open to feedback. With that caveat out of the way, here’s my initial solution:

; The range function here behaves as I would expect it to. Hooray!
(reduce +
  (filter 
    #(or (= 0 (mod % 3)) (= 0 (mod % 5)))
    (range 3 1000)))

You can plug this and future examples into the online REPL to verify them if you don’t happen to have Clojure installed. I have to admit, the lisp syntax is growing on me. The first time I looked at all of those parentheses I found myself walking to the kitchen for a glass of wine, but now I’m starting to find a certain beauty in them. Anyway, this first solution is analagous to the slowest C# solution above, which we know isn’t an efficient method for dealing with the problem. Let’s see if we can do any better:

(defn threes-and-fives [limit]
  (letfn [(multiplier [n acc]
    (let [three (* 3 n) five (* 5 n)]
      (if (< three limit)        
        (recur (inc n) (if (or (>= five limit) (zero? (mod five 3))) (+ three acc) (+ three five acc)))
        acc
      )))]
    (multiplier 1 0)))

(threes-and-fives 1000)

This looks and feels ghoulish so I’m going to assume it’s far from idiomatic(the names alone are too long given the Clojure I’ve seen). However, it benchmarks a heck of a lot better. Speaking of which, I had no idea how to benchmark Clojure, but some quick DDGing yielded the Criterium library which was easy to get up and running even for a total novice. Assuming you have Leiningen 2.0+ but haven’t set up any global plugins yet, you can just:

echo {:user {:plugins [[criterium "0.4.1"]]}} > ~/.lein/profiles.clj

Then to benchmark the solutions:

(use 'criterium.core)
(bench (reduce + (filter #(or (= 0 (mod % 3)) (= 0 (mod % 5))) (range 3 1000))))
(bench (threes-and-fives 1000))

Anyway, on to the benchmarks:

Note that the benchmarks aren’t comparable between languages as they’re running on totally different machine configurations. As expected, the second solution is much quicker than the first. I’m sure there’s a way to improve on it more. If you have one(or any other comments), let me know on Twitter.

Project Euler Problem 1 was originally published by Bret Koppel at Bret Koppel's Blog on August 21, 2013.

Cache Your XML Serializers

2012-07-13T00:00:00-04:00

“The Site is Slow”

We’d been getting reports about “slow” performance for a short time, but our feature list kept us from really looking into it. Finally, a primary client contact complained triggering a series of performance optimizations. In this post, I’ll explore one of the culprits: failing to cache our XML serializers.

Finding the Problem

A quick profile of network traffic on our pages revealed a group of sluggish AJAX calls. Generally this points to a database issue, but profiling the database didn’t back that up - Everything was coming back quickly, but wasn’t making it to the UI for ages. Something was going wrong in our code.

ANTS Marching

My first stop was the built-in Visual Studio Profiling Tools. Unfortunately, they don’t work out of the box with IIS Express. I fiddled with setup for a bit, but in the end I went with ANTS Profiler because they’re clear and easy to use out of the box. A few runs through ANTS pointed me to our serialization methods.

Everything looked kosher in the code. Some quick searches told me to use Sgen or switch to another serializer, like protobuf. Sgen didn’t provide the boost I was hoping for, and swapping serializers was going to be a headache. Frustrated, I shut down for the night and went home.

Cache Everything

One of my searches eventually lead me to Scott Hanselman’s post on caching XML serializers. Could something that plagued us in .NET 1.1 still be around in .NET 4? Sure enough, a little reflecting showed that the current XML serializer still only caches calls to two constructors: XmlSerializer(Type) and XmlSerializer(Type, String). Unfortunately, we use the XmlSerializer(Type, Type[]) constructor extensively.

A bunch of refactoring later, I ended up with the following:

public static XmlSerializer GetSerializer(Type type, Type[] extraTypes = null)
{
  if (type == null)
  {
    throw new ArgumentNullException("type");
  }

  int hash = 17;
  unchecked
  {
    hash = (hash * 17) + type.GetHashCode();
    if (extraTypes != null)
    {
      foreach (var extraType in extraTypes)
      {
          hash = (hash * 17) + extraType.GetHashCode();
      }
    }
  }

  var cacheName = "XmlSerializer-" + hash;
  var cachedSerializer = MemoryCache.Default[cacheName];
  if (cachedSerializer != null)
  {
    return (XmlSerializer)cachedSerializer;
  }

  var serializer = (extraTypes == null || extraTypes.Length == 0) ? new XmlSerializer(type) : new XmlSerializer(type, extraTypes);
  var cachePolicy = new CacheItemPolicy { SlidingExpiration = TimeSpan.FromHours(1) };
  MemoryCache.Default.Set(cacheName, serializer, cachePolicy);
  return serializer;
}

This actually ends up over-caching, which will need to be cleaned up. If we don’t have any extra types, the serializer will cache itself so there’s no point in me taking up extra memory.

Results

The results were pretty impressive:

Less than a third of the time? I’ll take it!

Comments? Find me on Twitter.

Cache Your XML Serializers was originally published by Bret Koppel at Bret Koppel's Blog on July 13, 2012.