tag:blogger.com,1999:blog-43768993570929525422024-02-08T06:39:52.072-08:00The Eclectic EngineerUnknownnoreply@blogger.comBlogger17125tag:blogger.com,1999:blog-4376899357092952542.post-44352814850880027002011-03-10T23:55:00.000-08:002011-03-11T00:01:28.070-08:00Amazon AMI Linux RPMS for S3FSI've been playing around with SparkleShare, Amazon AMI Linux and S3 which required building a bunch of RPMs. I'm posting them here so that others don't have to.<br />
<br />
For S3FS:<br />
<ul><li><a href="http://www.matthewstump.com/rpms/fuse-2.8.5-8.5.amzn1.x86_64.rpm">fuse-2.8.5-8.5.amzn1.x86_64.rpm</a></li>
<li><a href="http://www.matthewstump.com/rpms/fuse-devel-2.8.5-8.5.amzn1.x86_64.rpm">fuse-devel-2.8.5-8.5.amzn1.x86_64.rpm</a></li>
<li><a href="http://www.matthewstump.com/rpms/fuse-libs-2.8.5-8.5.amzn1.x86_64.rpm">fuse-libs-2.8.5-8.5.amzn1.x86_64.rpm</a></li>
<li><a href="http://www.matthewstump.com/rpms/s3fs-1.40-1.x86_64.rpm">s3fs-1.40-1.x86_64.rpm</a></li>
</ul><br />
For OpenVPN:<br />
<ul><li><a href="http://www.matthewstump.com/rpms/openvpn-2.2.beta5-1.x86_64.rpm">openvpn-2.2.beta5-1.x86_64.rpm</a></li>
<li><a href="http://www.matthewstump.com/rpms/pkcs11-helper-1.07-2.amzn1.1.x86_64.rpm">pkcs11-helper-1.07-2.amzn1.1.x86_64.rpm</a></li>
<li><a style="color: #cccccc;" href="http://www.matthewstump.com/rpms/pkcs11-helper-devel-1.07-2.amzn1.1.x86_64.rpm">pkcs11-helper-devel-1.07-2.amzn1.1.x86_64.rpm</a></li>
</ul>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-4376899357092952542.post-82086132286818802792010-03-27T00:30:00.000-07:002010-04-03T23:00:06.080-07:00Fetching all keys for a column using Cassandra's Thrift APIFigured out how to get all of the Cassandra key/value pairs for a column using the thrift API. The documentation on top of the horrible naming scheme definitely doesn't make it easy to figure things out quickly. Below is an example in Python
<script type="syntaxhighlighter" class="brush: xml"><![CDATA[
<Keyspaces>
<Keyspace Name="MyUberSite">
<ColumnFamily CompareWith="UTF8Type" Name="Users" />
<ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
<ReplicationFactor>1</ReplicationFactor>
<EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
</Keyspace>
</Keyspaces>
]]></script>
<pre class="brush: py">
#/usr/bin/env python
from thrift import Thrift
from thrift.transport import TTransport
from thrift.transport import TSocket
from thrift.protocol.TBinaryProtocol import TBinaryProtocolAccelerated
from cassandra import Cassandra
from cassandra.ttypes import *
import time
socket = TSocket.TSocket("localhost", 9160)
transport = TTransport.TBufferedTransport(socket)
protocol = TBinaryProtocol.TBinaryProtocolAccelerated(transport)
client = Cassandra.Client(protocol)
keyspace = "MyUberSite"
user_uuid = "0ad503dd-2642-4a1e-9113-a75bfd183c34"
try:
transport.open()
print "UUID: ", user_uuid, "\n"
# Create a column (user record) who's name is a UUID.
# Add two key/value pairs: one for email, one for username
column_path = ColumnPath(column_family="Users", column=user_uuid)
client.insert(
keyspace,
"email",
column_path,
"email@example.com",
time.time(),
ConsistencyLevel.ZERO)
client.insert(
keyspace,
"username",
column_path,
"user_account_name",
time.time(),
ConsistencyLevel.ZERO)
# Which column family are we interested in querying.
column_parent = ColumnParent(column_family="Users")
# A slice dictates our start and stop for column names we are interested in.
# We only want records for one column (user) so we are going to set the
# start and stop to the same values which is the UUID of the user we
# created above.
slice_range = SliceRange(
start=user_uuid,
finish=user_uuid)
# Create our predicate using the range instantiated above.
predicate = SlicePredicate(slice_range=slice_range)
# We want all of the column's (user's) keys so we are going to specify an
# empty key range. If we wanted a subset of the columns keys we could
# specify that subset here. The range is from start to stop using the
# sort we specified when we declared our column family in storage-conf.xml.
key_range = KeyRange("", "")
# Perform the query and pring the results.
print "\n\n".join(
map(
repr,
client.get_range_slices(
keyspace,
column_parent,
predicate,
key_range,
ConsistencyLevel.ONE)))
except Thrift.TException, tx:
print 'Thrift: %s' % tx.message
finally:
transport.close()
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-34460563770110934162010-03-22T22:56:00.000-07:002010-03-22T23:05:51.518-07:00Using AWK to split syslog files by proc idWhen troubleshooting a multi-threaded server it's sometimes nice to be able to split up the resulting log file by process making it a little bit easier to figure out what's going on. In this instance I'm splitting the log files from PostgreSQL by process ID which happens to be field 4.
<pre class="brush: bash">
# input format: 2010-01-25 15:41:46 PST [6534]: [30900-1] LOG: duration: 0.243 ms statement: SELECT ...
awk '{print > "substr($4, 2, length($4) - 3)"}' enroll.log
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-39187570740904800742010-03-22T22:42:00.000-07:002010-03-22T22:52:29.181-07:00Using AWK to remove newlines from textThis is an AWK one liner that comes in handy. It will strip newlines from it's input replacing them with a single space. I often use it in conjunction with things such as svn, cut, sed and grep.
<pre class="brush: bash">
awk -v RS="\n" -v ORS=" " {print}
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-54922850730898078862010-03-22T20:47:00.000-07:002010-03-22T21:43:57.203-07:00Install Thrift on OSX Snow LeopardThese are my notes for installing <a href="http://incubator.apache.org/thrift/">Thrift</a> on OSX Snow Leopard. The two pre-requisites are XCode with the 10.4 SDK, which is not installed by default and <a href="http://www.macports.org/">MacPorts</a>. This is a mashup of <a href="http://wiki.apache.org/thrift/ThriftInstallationMacOSX">three</a> <a href="http://www.facebook.com/topic.php?uid=15325934266&topic=11245">similar</a> <a href="http://ltslashgt.com/2009/04/23/installing-thrift-mac-os-x/">guides</a>, none of which contained all of the necessary information to get up and running.
<pre class="brush: bash">
export LOG4J_PATH='/opt/local/share/java/jakarta-log4j.jar'
export THRIFT_URL='http://gitweb.thrift-rpc.org/'
export THRIFT_PARAMS='?p=thrift.git;a=snapshot;h=HEAD;sf=tgz'
sudo port install boost
sudo port install jakarta-log4j
sudo port install pkgconfig
sudo port install libtool
echo "thrift.extra.cpath = ${LOG4J_PATH}" > ~/.thrift-build.properties
curl "${THRIFT_URL}${THRIFT_PARAMS}" > thrift.tgz
tar xzf thrift.tgz
cd thrift
export CFLAGS="-arch i386 -m32"
export CXXFLAGS="$CFLAGS"
cp /opt/local/share/aclocal/pkg.m4 aclocal/
./bootstrap.sh
./configure --prefix=/opt/local
make
sudo make install
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-905389204515904822009-06-20T14:32:00.000-07:002009-06-20T15:20:38.865-07:00VGA Pixelclock in VerilogI've been teaching myself Verilog and just recently got VGA working. More info and full source soon.
<div style="width:320px;text-align:right;"><embed width="320" height="240" src="http://static.photobucket.com/flash/rss_slideshow.swf?rssFeed=http%3A%2F%2Ffeed291.photobucket.com%2Falbums%2Fll298%2Fmrevilgnome%2FVGA%2Ffeed.rss" type="application/x-shockwave-flash" wmode="transparent"></embed><a href="http://s291.photobucket.com/albums/ll298/mrevilgnome/VGA/" target="_blank"><img src="http://pic.photobucket.com/share/icons/embed/btn_viewall.gif" style="border:none;" /></a></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-67267604113221738942008-06-09T14:47:00.000-07:002008-06-09T15:18:25.282-07:00ISight Screen SaverA while ago I wrote a screen saver for OSX as a learning exercise. I wanted to learn how to access the camera built into my Macbook Pro and get my feet wet with OSX. I was going to integrate face recognition using the OpenCV or MPT libraries but never got around to it. I'm posting it now because some of the guys at work saw it and wanted a copy. I added the source to my <a href="http://code.google.com/p/mstump-learning-exercises/">SVN repository</a> and made a <a href="http://code.google.com/p/mstump-learning-exercises/downloads/list">binary available via the download site</a>.
The screen saver is really as minimalistic as it gets. It uses the new QTKit API for accessing the camera and sets the capture view to the frame size of the inherited ScreenSaverVeiw class. The QTKit API came out around the time of Leopard's release and is much simpler and has better performance then the old streamgrabber API which had been around since the OS 9 days.
To install the binary just download the zip file, unzip and double click. It will automatically install itself and become available as a screen saver in the system preferences.
<center><a href="http://i291.photobucket.com/albums/ll298/mrevilgnome/Picture1.png" target="_blank">
<img src="http://i291.photobucket.com/albums/ll298/mrevilgnome/th_Picture1.png" /></a>
</center>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-1206706453953193502008-06-08T16:57:00.000-07:002010-03-22T21:56:40.427-07:00Jarvis March Convex Hull Algorithm in F#I recently obtained the "Dutch"<a href="http://www.amazon.com/Computational-Geometry-Applications-Mark-Berg/dp/3540779736/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1212969817&sr=8-1"> computational geometry book</a> to supplement the other computational <a href="http://www.amazon.com/gp/product/0123694469">geometry book that I have</a>. Instead of doing what I normally do which is skip directly to the topic that I'm interested in, I've resolved to work through the entire book systematically implementing each of the algorithms in F# or some other functional language. The purpose is two fold; I know almost nothing about the topic and it's hard to find anything about computational geometry implementations in a functional language. Who knows, someone may actually find my floundering helpful.<br/><br/>
Chapter one is all about convex hulls and after doing a little bit of reading I determined that the "Jarvis March" algorithm looked like a good place to start. I've only included core part of the algorithm here in the blog post. The entire source including the visualization can be found <a href="http://mstump-learning-exercises.googlecode.com/svn/trunk/fsharp/convex_hull_jarvis/convex_hull.fsx">here</a>.<br/><br/>
The algorithm is pretty simple I start first by sorting the point list by the Y coordinate and use the minimum as the starting point for my hull. I start working through the list trying to find the left most point in relation to the previous point added to the hull; this is the fold_left bit of the _jarvis function. Once I've found the left most point I append it to the head of the hull list and make a recursive call to _jarvis. When I reach the point with the maximum Y value I know that I have successfully built the left side of the hull. I then repeat the entire process but this time instead of searching for the left most point I search for the right most point building up the right side of the hull. When done I append these two lists together to form the complete hull.<br/><br/>
<pre class="brush: fsharp">type Point = {
x :double;
y :double;
} with
static member ( = ) a b =
a.x = b.x && a.y = b.y;
static member ( <> ) a b =
not(a = b);
end;;
type Edge = {
orig:Point;
dest:Point;
} with
static member ( = ) a b =
a.orig = b.orig && a.dest = b.dest;
static member ( <> ) a b =
not(a = b);
end;;
let sort point_list =
List.sort( fun p1 p2 ->
match Pervasives.compare p1.y p2.y with
| 0 -> Pervasives.compare p1.x p2.x;
| _ as v -> v;
) point_list;;
// Input: three points P0, P1, and P2
// Return: >0 for P2 left of the line through P0 and P1
// =0 for P2 on the line
// <0 for P2 right of the line
let testSide p0 p1 p2 =
(p1.x - p0.x) * (p2.y - p0.y) - (p2.x - p0.x) * (p1.y - p0.y);;
let isLeft p0 p1 p2 =
(testSide p0 p1 p2) > 0.;;
let isRight p0 p1 p2 =
(testSide p0 p1 p2) < 0.;;
let rec _jarvis cmp max_point (hull_list:Point list) (input_list:Point list) =
match input_list with
| p::tl ->
let last = List.hd hull_list in
if last = max_point then hull_list
else (
let best = List.fold_left(fun p1 p2 ->
if (cmp last p1 p2) then p2
else p1;
) p input_list in
_jarvis cmp max_point (best::hull_list) tl
)
| [] -> hull_list;;
let jarvis point_list =
if List.length point_list < 3 then failwith "3 points are required for a polygon";
let point_list = sort point_list in
let miny = point_list |> List.hd in
let maxy = point_list |> List.rev |> List.hd in
let left = _jarvis (isLeft) maxy [miny] point_list in
let right = _jarvis (isRight) maxy [miny] point_list in
right @ left |> List.rev;;
</pre>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-4376899357092952542.post-65764552994692549552008-05-18T10:40:00.000-07:002008-12-11T18:10:02.344-08:00XNA, Boids and Buffalo! Oh My!<span style="font-family:arial;">So I've spent my time since my last post basically re-writing the entire Boids project. I've thrown out the GDI UI and have replaced it with an <a href="http://creators.xna.com/">XNA</a> based one. I also rewrote my Boid type to be a class which inherits from the Body type in the <a href="http://www.codeplex.com/FarseerPhysics">Farseer</a> 2D physics engine, making the simulation much more realistic. I’ve now got mass, rotation, torque, collisions, drag and friction to play with.</span><br/><br/>
<span style="font-family:arial;">I've also refined and rewritten most of my rules, fixing the stationary swarm problem, and have added more random noise making things much more interesting. Random Boids will peel off from the group, wander around and then later rejoin. Sometimes the herd separates into two or more sub-groups then later coalesces back into a single herd. There is definitely some additional tweaking of the rules to be done, but I’m definitely headed in the right direction.</span><br/><br/>
<span style="font-family:arial;">Oh, and this is one of my favorite parts, I replaced the dots that represented the Boids in the old simulations with little buffalo which rotate to reflect the actual heading of the Boid. So I have a little herd of buffalo roaming around in my computer, for some reason that makes me very happy.</span><br/><br/>
<span style="font-family:arial;">This being a learning exercise, I separated the F# rules and logic into multiple files so that I could play with modules and namespaces, and have now tackled almost every subject in the <a href="http://www.amazon.com/Expert-F-Experts-Voice-Net/dp/1590598504">Expert F#</a> book. I don’t quite live in F# yet, but I’m getting much faster and am programming with fewer mistakes which is a good sign.</span><br/><br/>
<span style="font-family:arial;">Playing with XNA was also quite fun. I am by no means utilizing all of the functionality provided, but from the limited area that I have touched it appears that Microsoft did a pretty good job. I played with OpenGL a couple of years ago, creating a planetary body orbit/gravity simulation with my roommate, and I remember it being painful. With XNA this was definitely not the case, it was very simple to get something up and running quickly. I am by no means a Microsoft fan-boy but you have to give them credit, they do make nice development tools.</span><br/><br/>
<span style="font-family:arial;">So where do I go next? I need to do some mundane things like tweak the existing rules, and port all of the XNA code from C# to F#. I also need to go through and clean up some of the code, I know I’m not doing several things very efficiently and as a result my laptop has trouble with 100 or more Boids. I also want to implement several more rules: herd leaders, predator/prey behavior, obstacle avoidance etc… I still want to rewrite the nearest neighbor search portion of the code, but that is slower going. One of the problems of being a college drop-out is that I never took discreet math or linear algebra, so understanding some of the algorithms takes a little bit more time than I anticipated. It’s not exactly hard, just requires more research on my part but it’s good for me. Also now that I’ve been reading <a href="http://www.amazon.com/gp/product/0123694469">computational geometry books</a> I see solutions to problems that I encounter all the time, I just wasn’t aware of the body of work that existed. I don’t feel too guilty though, I don’t think any of these algorithms were taught at an undergraduate level anyways at the U of Arizona. Who knows, when I was there 2001-2004 the program was in a real state of decline and was rife with in-fighting and I was all too eager to leave.</span><br/><br/>
<span style="font-family:arial;"><span style="font-weight: bold;">FYI:</span> The project has gotten too big to keep posting code in the blog, so I created a <a href="http://code.google.com/p/fsharp-boids/">project</a> on <a href="http://code.google.com/">Google Code</a>.</span>
<center>
<a href="http://i291.photobucket.com/albums/ll298/mrevilgnome/boids2.png" target="_blank">
<img src="http://i291.photobucket.com/albums/ll298/mrevilgnome/th_boids2.png" /></a>
</center>Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-4376899357092952542.post-2521811748768786652008-05-11T22:07:00.000-07:002010-03-22T22:10:55.495-07:00Boids: Revision TwoA friend of mine was in town this week, and haven't had much spare time to play with my side projects. When I finally got down to it I was able to redo my boids simulation to use operator overloading, boundary rules, and improved some of the rendering. I based my implementation off of a couple of different sources: a <a href="http://www.vergenet.net/%7Econrad/boids/pseudocode.html">pseudo-code explanation</a> of the original boids paper<a href="http://www.vergenet.net/%7Econrad/boids/pseudocode.html"></a>, a <a href="http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/502240">Python implementation</a>, and <a href="http://www.koders.com/java/fidE8F468B83DF5CCD939CB4AB0675E36070DA41FA3.aspx?s=boid#L7">this Java implementation</a>. There is one remaining problem that I have yet to figure out. I originally used the rules laid out in the Python implementation, but the animation was very jerky when rule 3 took affect, and once the dots swarmed together they just sit their stationary. Finding no difference between his implementation and my own I started looking around for other examples that might give me an insight. That is when I found the previously mentioned Java implementation which resulted in a much more smooth trajectory. Unfortunately it still did not fix the problem where after swarming together the boids don't go anywhere. I'm going to continue to work on it but wanted to post my results thus far.
<pre class="brush: fsharp">
open System;;
open System.Windows.Forms;;
open Microsoft.FSharp.Collections;;
let BOID_COUNT = 40;;
let HEIGHT = 700.;;
let WIDTH = 1000.;;
let WALL = 50.;;
let WALL_FORCE = 30.;;
let MAX_VEL = 800.;;
let R1_CONST = 0.2;;
let R2_CONST = 0.2;;
let R2_RANGE = 100.;;
let R3_CONST = 0.2;;
let FPS = 24;;
let RECT_HW = 3;;
type vector = {
x : double;
y : double;
} with
member v.mag() =
(v.x ** 2.0) + (v.y ** 2.0) |> Math.Sqrt;
static member ( + )(a:vector, b:vector) =
{x = a.x + b.x;
y = a.y + b.y};
static member ( - )(a:vector, b:vector) =
{x = a.x - b.x;
y = a.y - b.y};
[<OverloadID("vect_mul_int")>]
static member ( * ) (a:vector, b:int) =
let bd = Convert.ToDouble(b) in
{x = a.x * bd;
y = a.y * bd};
[<OverloadID("vect_mul_doub")>]
static member ( * ) (a:vector, b:double) =
{x = a.x * b;
y = a.y * b};
[<OverloadID("vect_pow_int")>]
static member ( ** ) (a:vector, b:int) =
let bd = Convert.ToDouble(b) in
{x = a.x ** bd;
y = a.y ** bd};
[<OverloadID("vect_pow_doub")>]
static member ( ** ) (a:vector, b:double) =
{x = a.x ** b;
y = a.y ** b};
[<OverloadID("vect_div_int")>]
static member ( / ) (a:vector, b:int) =
let bd = Convert.ToDouble(b) in
{x = a.x / bd;
y = a.y / bd};
[<OverloadID("vect_div_double")>]
static member ( / ) (a:vector, b:double) =
{x = a.x / b;
y = a.y / b};
static member ( > ) (a:vector, b:double) =
a.mag() > b;
static member ( < ) (a:vector, b:double) =
a.mag() < b;
end;;
type boid = {
pos : vector;
vel : vector;
} with
static member ( + )(a:boid, b:boid) =
{pos = a.pos + b.pos;
vel = a.vel + b.vel};
static member ( - )(a:boid, b:boid) =
{pos = a.pos - b.pos;
vel = a.vel - b.vel};
member b.distance b2 =
Math.Sqrt(Math.Abs(((b.pos.x - b2.pos.x) ** 2.0) - ((b.pos.y - b2.pos.y) ** 2.0)));
member b.update_pos() =
{b with
pos = b.pos + b.vel};
member b.neighbors(r, bl) =
[for n in bl ->
(b.distance(n), n)] |>
List.filter(fun (d:double, _) -> d < r) |>
List.sort(fun (d1:double, _) (d2:double, _) -> d1.CompareTo(d2)) |>
List.tl |>
List.unzip;
end;;
let rand = new Random();;
let rand_double(max:double) =
Convert.ToDouble(rand.Next(Convert.ToInt32(max) - 1)) + rand.NextDouble();;
let vector_init =
{x = 0.;
y = 0.};;
let boid_init =
{pos = vector_init;
vel = vector_init};;
let boid_sum_list bl =
List.fold_left(+) boid_init bl;;
let boid_adjust_v_to_fps b =
{ b with
vel = b.vel / (Convert.ToDouble(FPS))};;
let boid_limit_vel b =
let v = b.vel.mag() in
match v with
| _ when v > MAX_VEL ->
let vl = v / MAX_VEL in
{ b with
vel = b.vel / vl}
| _ -> b;;
let boid_check_x_bounds b =
match b with
| _ when b.pos.x < WALL ->
{b with
vel = {b.vel with
x = b.vel.x + WALL_FORCE}};
| _ when b.pos.x > (WIDTH - WALL) ->
{b with
vel = {b.vel with
x = b.vel.x - WALL_FORCE}};
| _ -> b;;
let boid_check_y_bounds b =
match b with
| _ when b.pos.y < WALL ->
{b with
vel = {b.vel with
y = b.vel.y + WALL_FORCE}};
| _ when b.pos.y > (WIDTH - WALL) ->
{b with
vel = {b.vel with
y = b.vel.y - WALL_FORCE}};
| _ -> b;;
let boid_rule1 b_sum count =
fun (b) ->
{b with
vel = b.vel + ((((b_sum.pos - b.pos) * (1. / (count - 1.))) - b.pos) * R1_CONST) };;
let boid_rule2 bl count =
fun (b:boid) ->
let (_, nbl) = b.neighbors(R2_RANGE, bl) in
let r2b = List.fold_left(fun b1 b2 -> b1 - (b2 - b)) boid_init nbl in
{ b with
vel = b.vel + ((r2b.pos * (1. / (count - 1.))) * R2_CONST) };;
let boid_rule3 b_sum count =
fun b ->
{b with
vel = b.vel + ((((b_sum.vel - b.vel) * (1. / (count - 1.))) - b.vel) * R3_CONST) };;
let boid_apply_rules b bl b_sum count =
b |> boid_check_y_bounds |> boid_check_x_bounds |> boid_rule1 b_sum count |> boid_rule2 bl count |>
boid_rule3 b_sum count |> boid_limit_vel |> boid_adjust_v_to_fps |> fun b -> b.update_pos();;
let get_wall() =
let height = Convert.ToInt32(HEIGHT) in
let width = Convert.ToInt32(WIDTH) in
let wl = Convert.ToInt32(WALL) in
[|new Drawing.Point(wl, wl);
new Drawing.Point(wl, height - wl);
new Drawing.Point(width - wl, height - wl);
new Drawing.Point(width - wl, wl)
|];;
type BoidControl = class
inherit UserControl as base
val mutable boidList:(boid list);
val brap:(Drawing.Point array);
val count:double;
val timer:Timer;
new(bl:(boid list)) as this = {
boidList = bl;
count = bl |> List.length |> Convert.ToDouble;
timer = new Timer();
brap = get_wall();
} then this.init();
member this.init() =
let height = Convert.ToInt32(HEIGHT) in
let width = Convert.ToInt32(WIDTH) in
let size = new Drawing.Size(width, height) in
base.Width <- width;
base.Height <- height;
base.MinimumSize <- size;
base.MaximumSize <- size;
base.BackColor <- Drawing.Color.White;
base.SetStyle (ControlStyles.UserPaint, true);
base.SetStyle (ControlStyles.DoubleBuffer, true);
base.SetStyle (ControlStyles.AllPaintingInWmPaint, true);
this.timer.Interval <- FPS;
this.timer.Tick.Add(fun _ ->
this.tick(););
this.timer.Start();
member this.boids_update() =
let b_sum = boid_sum_list this.boidList in
let al = [for b in this.boidList -> async { return boid_apply_rules b this.boidList b_sum this.count }] |> Async.Parallel in
this.boidList <- (Async.Run(al) |> Array.to_list);
();
override this.OnPaint e =
let g = e.Graphics in
List.iter(fun b ->
let x = Convert.ToInt32(b.pos.x) in
let y = Convert.ToInt32(b.pos.y) in
let bp1 = new Drawing.Point(x - RECT_HW, y - RECT_HW) in
let bp2 = new Drawing.Point(x - RECT_HW, y + RECT_HW) in
let bp3 = new Drawing.Point(x + RECT_HW, y + RECT_HW) in
let bp4 = new Drawing.Point(x + RECT_HW, y - RECT_HW) in
g.FillPolygon(Drawing.Brushes.Red, [|bp1; bp2; bp3; bp4|]);) this.boidList;
g.DrawPolygon(Drawing.Pens.Blue, this.brap);
member this.tick() =
this.boids_update();
this.Invalidate();
end;;
[<STAThread()>]
do
let height = Convert.ToInt32(HEIGHT) in
let width = Convert.ToInt32(WIDTH) in
let size = new Drawing.Size(width, height) in
let bl = [for x in 1..BOID_COUNT -> {pos = {x = 50. + rand_double(WIDTH - 50.);
y = 50. + rand_double(HEIGHT - 50.)};
vel = vector_init }] in
let mw = new Form() in
let bc = new BoidControl(bl) in
mw.Controls.Add(bc);
mw.ClientSize <- size;
mw.MaximumSize <- mw.Size;
mw.MinimumSize <- mw.Size;
mw.Text <- "Boids";
mw.Show();
Application.Run(mw);;
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-14999637693400210232008-05-05T19:12:00.000-07:002008-05-05T19:15:50.836-07:00Boids!I've got revision one of my boids simulation done. I'll post more details and code later, but for now here is screen capture.
<center>
<a href="http://i291.photobucket.com/albums/ll298/mrevilgnome/boids1.jpg" target="_blank">
<img src="http://i291.photobucket.com/albums/ll298/mrevilgnome/th_boids1.jpg"></a>
</center>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-27222120072493699202008-04-30T11:58:00.001-07:002010-03-22T22:13:43.382-07:00Asynchronous workflows in F#Work has been really busy and I haven't had time to work on any of my side projects until last night, when I had chance to play with asynchronous workflows in F#. I've been wanting to explore to a greater extent what is out there in the world of concurrent and distributed computing, but in order to get started I needed to find some project that would be the catalyst. Not that having a project is absolutely necessary but it helps keep me on task and focused towards an end goal. Some of the ideas I've been tossing around were creating an engine to decrypt/encrypt PGP MIME messages, or build a better load/stress engine, both of which require a large up-front investment of time.<br/><br/>
After about a week of going nowhere I stumbled upon the idea of doing swarm animations, and possibly throw in some additional AI behavior if I had time. The thing about doing swarm animations is that each entity in the swarm needs to move in relation to each of it's neighbors which involves a nearest neighbor search, one of those computationally costly and hard to tackle problems. I'm going to start with a really naive brute force approach and evolve it over time, into something more efficient. One of the things that I've been looking at is the Stolfi and Guibas method for creating Voronoi diagrams in order to speed up the nearest neighbor search. It's a divide an conquer algorithm so it should lend itself to parallelization, but that comes later, today I start with asynchronous workflows. The bit of code below had the following runtime on my Dell D620 dual-core laptop when executed using fsi:
<pre>
Async time: 00:00:34.7309424
Sync time: 00:00:40.1761800
</pre>
<pre class="brush: fsharp">
let largest = 800;;
type point = {
x : double;
y : double;
}
let rand = new Random();;
let rand_double(max) =
System.Convert.ToDouble(rand.Next(max - 1)) + rand.NextDouble();;
let rec quick_sort(x) =
match x with
| pivot :: rest ->
let left, right = List.partition (( > ) pivot) rest in
quick_sort left @ pivot :: quick_sort right
| [] -> [];;
let distance(p1, p2) =
Math.Sqrt(Math.Abs(((p1.x - p2.x) ** 2.0) - ((p1.y - p2.y) ** 2.0)));;
let time(f) =
let start = DateTime.Now in
f();
DateTime.Now - start;;
let neighbors(p, pl) =
[for n in pl -> (distance(n, p), n)];;
let async_neighbors(p, pl) =
let a = Async.Parallel[for n in pl -> async {return (distance(n, p), n)}] in
Async.Run(a);;
let sync_test() =
let pl = [for x in 1..5000 -> {x=rand_double(largest);y=rand_double(largest)}] in
[for p in pl -> neighbors(p, pl)] |> ignore;;
let async_test() =
let pl = [for x in 1..5000 -> {x=rand_double(largest);y=rand_double(largest)}] in
let a = Async.Parallel[for p in pl -> async { return neighbors(p, pl) }] in
Async.Run(a) |> ignore;;
let async_test2() =
let pl = [for x in 1..5000 -> {x=rand_double(largest);y=rand_double(largest)}] in
let a = Async.Parallel[for p in pl -> async { return async_neighbors(p, pl) }] in
Async.Run(a) |> ignore;;
let cleanup() =
GC.Collect();
GC.WaitForPendingFinalizers();;
do Console.WriteLine("Async time: " + any_to_string(time(async_test)));;
do cleanup();;
do Console.WriteLine("Sync time: " + any_to_string(time(sync_test)));;</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-58464497406137245032008-04-03T21:28:00.000-07:002010-03-22T22:14:30.845-07:00Calling asynchronous Twisted Python code from a blocking functionSo I had to write a VAPI-XP test for Mercury/HP Quality Center, and the way that this particular test type works is that you write a script in Python, VBScript, or Perl. This script must have a TestMain function that is called and as soon as that function exits the test is over and everything exits. This test had to SSH into one of the machines in our virtualized test bed, and tell the machine to netboot in order to pick up the newest build of our server product.<br/><br/>
I could have just used a call to os.popen() to some SSH client on our intermediary testing host but I had have bad experiences with that in the past, mostly weird behavior with OpenSSH and plink. I had written some SSH code with the <a href="http://twistedmatrix.com/trac/">Twisted</a> module <a href="http://twistedmatrix.com/projects/conch/documentation/howto/conch_client.html">Conch</a> before so I decided to just go ahead and use that. So got the SSH code up and running no problem, but how was I to block execution until the event was triggered letting me know that the session was complete? Furthermore how was I going to get my data back without doing something dirty like a global variable?<br/><br/>
I talked to some guys on IRC and came up with the solution below. The SSH code itself isn't that interesting and I've only really included it for completeness the important bits to note is the call to "threads.blockingCallFromThread" in RunSSHCommand and note that I'm passing the deferred that I create in "_runSSHCommand" through each of the SSH classes (usually 'd' or 'self._d') and that it is finally being called in the method "CommandChannel.closed".
<pre class="brush: python">
#!/usr/bin/python
class ClientTransport(transport.SSHClientTransport):
def __init__(self, user, command, d):
self._user = user
self._command = command
self._d = d
def verifyHostKey(self, pubKey, fingerprint):
#we don't care accept any host key
return defer.succeed(1)
def connectionSecure(self):
#connection made, instiantiate the class that will handle authentication
#and pass an instance of the class that embodies the user behavior as a param
self.requestService(ClientUserAuth(self._user, ClientConnection(self._command, self._d)))
class ClientUserAuth(userauth.SSHUserAuthClient):
def getPassword(self, prompt = None):
# this says we won't do password authentication
return
def getPublicKey(self):
#return the public key which is defined up top as a string
return keys.getPublicKeyString(data=SSH_PUB_KEY)
def getPrivateKey(self):
#return the pricate key which is also defined up top as a string
return defer.succeed(keys.getPrivateKeyObject(data=SSH_PRIV_KEY))
class ClientConnection(connection.SSHConnection):
def __init__(self, cmd, d, *args, **kwargs):
connection.SSHConnection.__init__(self)
self._command = cmd
self._d = d
def serviceStarted(self):
self.openChannel(CommandChannel(self._command, self._d, conn=self))
class CommandChannel(channel.SSHChannel):
name = 'session'
def __init__(self, command, d, *args, **kwargs):
channel.SSHChannel.__init__(self, *args, **kwargs)
self.command = command
self.d = d
self.data = ""
def channelOpen(self, data):
#send an execute request passing the user supplied string.
self.conn.sendRequest(self,
'exec',
common.NS(self.command),
wantReply=True
).addCallback(self._gotResponse)
def _gotResponse(self, _):
#command has returned, send an EOF in order to terminate the connection
self.conn.sendEOF(self)
def dataReceived(self, data):
#append the output data to the string
self.data = self.data + data
def closed(self):
#connection closed, execute the callback on the deferred
#that we have been passing around
self.d.callback(self.data)
class ClientCommandFactory(protocol.ClientFactory):
#factory class for our SSH protocol
def __init__(self, user, command, d):
self._user = user
self._command = command
self._d = d
def buildProtocol(self, addr):
protocol = ClientTransport(self._user, self._command, self._d)
return protocol
def RunSSHCommand(hostname, port, user, cmd):
#Run an SSH command as a user on a given host, we only use key authentication
thread.start_new_thread(reactor.run, (False,))
try:
return threads.blockingCallFromThread(reactor, _runSSHCommand, hostname, port, user, cmd)
except:
return None
reactor.stop()
def _runSSHCommand(hostname, port, user, cmd):
d = defer.Deferred()
factory = ClientCommandFactory(user, cmd, d)
reactor.connectTCP(hostname, 22, factory)
return d
if __name__ == "__main__":
RunSSHCommand(hostname, port, user, "register_netboot")
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-76576826971633691872008-04-03T15:46:00.001-07:002008-04-03T20:55:14.022-07:00Formatting code for HTMLThis being my first blog I didn't have any tricks in my bag for formatting code. After some Google searching I only found two projects that looked useful. The first being <a href="http://formatmysourcecode.blogspot.com/">this project</a> which wasn't language aware and couldn't do any syntax highlighting. The second more promising project was <a href="http://www.manoli.net/csharpformat/">this one</a> and fortunately he posted the C# source for his library. Only problem is that it didn't support F# or Python. I went ahead and added support for those two languages, created a front end for the thing and added support for user specified <span class="blsp-spelling-error" id="SPELLING_ERROR_0">CSS</span> documents. In addition to exposing most of the built-in functionality my app has a preview mode which will display the formatted code in an embedded IE pane. I've posted my version of the <a href="http://www.eclecticengineers.com/FormatCode.zip"><span class="blsp-spelling-corrected" id="SPELLING_ERROR_1">library</span> here</a>, and I've also made available a <a href="http://www.eclecticengineers.com/FormatCodeBinary.zip"><span class="blsp-spelling-error" id="SPELLING_ERROR_2">pre</span>-compiled binary</a>.
<center>
<a href="http://i291.photobucket.com/albums/ll298/mrevilgnome/preview.png" target="_blank">
<img src="http://i291.photobucket.com/albums/ll298/mrevilgnome/th_preview.png" /></a>
<a href="http://i291.photobucket.com/albums/ll298/mrevilgnome/output.png" target="_blank">
<img src="http://i291.photobucket.com/albums/ll298/mrevilgnome/th_output.png" /></a>
<a href="http://i291.photobucket.com/albums/ll298/mrevilgnome/input.png" target="_blank">
<img src="http://i291.photobucket.com/albums/ll298/mrevilgnome/th_input.png" /></a>
</center>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-2821388948488305622008-03-31T17:11:00.000-07:002010-03-22T22:17:23.713-07:00The no-frills guide to .Net COM interopSo there a couple of guides out there that already cover some aspects of this post, but none of them cover all of the topics and many of them are convoluted. When I publish something to COM I usually end up consuming it via a scripting language like Python, Perl or VBScript so this little howto will reflect what I have found to be best practices. So here is the no-frills guide to exposing a C# class to COM.
<ul><li>Create an interface</li><li>Create a class that inherits from the interface</li><li>Add GUIDs to both interface and class using the "Guid" attribute define in "System.Runtime.InteropServices". You can use the GUID generation tool found in the "Tools" menu of Visual Studio.</li></ul>
<pre class="brush: csharp">
using System;
using System.Runtime.InteropServices;
namespace MyNamespace{
[Guid("EFB47A26-D84F-4092-927D-232E92FB77E8")]
public interface IMyComponent{
[DispId(1)]
void DoStuff();
}
[Guid("8F3DFCDB-45F7-461d-9B19-B140CD6F05FA")]
public class MyComponent : IMyComponent{
public void DoStuff() { }
}
}
</pre>
<ul><li><span>Add the interface type to the interface, you don't need to know all the details but you should probably use the Dispatch type if this component is being consumed by a dynamically typed language such as Python.</span></li></ul><ul><span><li>Add the DispId to each method exposed by the interface. Again this helps when being consumed by Python. A note of caution though, that once these values are set and you register the component they should not change. It's not the end of the world if they do, but you will cause headaches for yourself.</li></span></ul>
<pre class="brush: csharp">
[Guid("EFB47A26-D84F-4092-927D-232E92FB77E8")]
[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
public interface IMyComponent {
[DispId(1)]
void DoStuff();
}
</pre>
<span><ul><li>Add the ProgId attribute to the class, set the value to Namespace.ClassName. This is again one of those things that may not be necessary but helps when using Python, Perl etc...</li><li>Add the ClassInterface attribute to the class setting the value to "ClassInterfaceType.AutoDispatch"</li><li>Add the ComDefaultInterfaceAttribute setting the value to the type of your interface. Again, not neccessarily required but Python has all sorts of trouble generating the wrapper classes if it isn't present.</li></ul></span>
<pre class="brush: csharp">
[ProgId("MyNamespace.MyComponent")]
[ClassInterface(ClassInterfaceType.AutoDispatch)]
[ComDefaultInterfaceAttribute(typeof(IMyComponent))]
[Guid("8F3DFCDB-45F7-461d-9B19-B140CD6F05FA")]
public class MyComponent : IMyComponent {
public void DoStuff() { }
}
</pre>
You need to enable two options in Visual Studio for your project. Open the properties for your project, and go the the application tab. Click on the "Assembly Information" button, and enable the make COM visible option. On the build tab you will also need to enable the register for com interop option at the bottom. You will also need to sign your assembly; you can use a self generated key through the signing tab of your project's properties.
<span>
Once compiled Visual Studio will auto-register the assembly with COM so that you can test the interface with a script or OLEView. You can use regasm or build a Wix project to have it installed permanently.</span>
<pre class="brush: csharp">
using System;
using System.Runtime.InteropServices;
namespace MyNamespace
{
[Guid("EFB47A26-D84F-4092-927D-232E92FB77E8")]
[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
public interface IMyComponent{
[DispId(1)]
void DoStuff();
}
[ProgId("MyNamespace.MyComponent")]
[ClassInterface(ClassInterfaceType.AutoDispatch)]
[ComDefaultInterfaceAttribute(typeof(IMyComponent))]
[Guid("8F3DFCDB-45F7-461d-9B19-B140CD6F05FA")]
public class MyComponent : IMyComponent{
public void DoStuff() { }
}
}
</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-17877270372870167842008-03-28T10:49:00.000-07:002010-03-22T22:15:36.419-07:00Playing around in WMI with F#Started playing around with WMI in F# in order to update DNS records. After several hours I finally got it working. I ran into some problems on the last line of the function below. Apparently function used with iter() must return "unit" which is the F# equivalent of null. My anonymous function was returning an obj, appending the "; ()" to the end of the function fixed the return type. "()" in F# is the notation for the unit type. The guys from F# hub said this could also be done by using "|> ignore".
<pre class="brush: fsharp">
let UpdateCNameDNSRecord(record:string, value:string) =
let query = String.Format("SELECT * FROM MicrosoftDNS_CNAMEType WHERE OwnerName='{0}'", record) in
let scope = (@"\\.\root\MicrosoftDNS") in //new ManagementScope
let searcher = new ManagementObjectSearcher(scope, query) in
if (not (value.EndsWith("."))) then
let value = (value + ".") in
let inputP = [|null; box(value)|] in
IEnumerable.iter(
fun (x:ManagementObject) ->
if (x.Item("PrimaryName").ToString() <> value) then
x.InvokeMethod("Modify", inputP) |> ignore
) (IEnumerable.untyped_to_typed(searcher.Get()));;</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-4376899357092952542.post-56355763470322851252008-03-26T10:51:00.000-07:002010-03-22T22:18:13.199-07:00An example win32 service in F#<span style="font-size:100%;">I needed a small single purpose <span class="blsp-spelling-error" id="SPELLING_ERROR_0">TCP</span> socket server for work, and I thought it might be an opportune time to try a new programming language. My choices were <span class="blsp-spelling-error" id="SPELLING_ERROR_1">Scala</span>, F# or <span class="blsp-spelling-error" id="SPELLING_ERROR_2">Erlang</span>. The target platform was a win2k box and I wanted to be able to register it as a service so F# was the obvious choice. In the code below I modified the <a href="http://stuff.mit.edu/afs/athena/software/fsharp_v1.1.12/FSharp-1.1.12.3/samples/fsharp/Sockets/Doc.html">echo server</a> example from the <a href="http://stuff.mit.edu/afs/athena/software/fsharp_v1.1.12/FSharp-1.1.12.3/README-fsharp.html">MIT F# site</a> to run as a service. </span>
<pre class="brush: fsharp">
// This sample code is provided "as is" without warranty of any kind.
// We disclaim all warranties, either express or implied, including the
// warranties of merchantability and fitness for a particular purpose.
#light
#r "System.ServiceProcess.dll";;
#r "System.Configuration.Install.dll";;
// Echo Server: opens a service on the localhost which bounces data straight back
open System.Net
open System.Net.Sockets
open System.Threading
open System.ServiceProcess
open System.ComponentModel
open System.Configuration.Install
open Microsoft.FSharp.Compatibility.CompatArray
let spawn f = (new Thread(new ThreadStart(f))).Start()
type EchoServer = class
inherit ServiceBase as base
new() as this = {} then base.ServiceName <- "SimpleEchoService"
override this.OnStart(args:string[]) =
spawn(this.listen)
member this.listen() =
let sock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.IP)
sock.Bind(new IPEndPoint(IPAddress.Any, 5120))
let backlog = 2 // number of incoming connections that can be queued for acceptance
sock.Listen(backlog)
let port = (sock.LocalEndPoint :?> IPEndPoint).Port
Printf.printf "server accepting on port %d...\n" port
while (true) do
let csock = sock.Accept()
Printf.printf "the server has accepted a client on port %d...\n" port;
spawn(fun () ->
let total = ref 0
try
let buf = Bytearray.create 64
let stream = new NetworkStream(csock)
while true do
Printf.printf "server reading...\n";
let nread = stream.Read(buf,0,64)
Printf.printf "server read %d...\n" nread;
stream.Write(buf,0,nread);
total := !total + nread;
done;
with
| :? Sockets.SocketException
| :? System.IO.IOException ->
Printf.printf "client gone: server read %d bytes from that client\n" !total;
flush stdout
)
done
end</pre>Unknownnoreply@blogger.com2