The Smallest Agile Methodology That Could Possibly Work

People

You need 3 hats. That might mean 3 people, or 3 groups of people, or 1 or 2 people wearing more than 1 hat:

  • 1 person who knows what they want to have.
  • 1 person who knows about how to provide that.
  • 1 person with enough money to pay for it to be provided.

People Checklist

1) Do you have someone to wear each of the 3 hats?

A: I don't have someone who knows what they want.

  • Consider a research project instead, in which you try to work out what you want.
  • Otherwise Stop Now.

A: I don't have someone who knows how to provide what I want

  • Find help. Otherwise Stop Now.

A: I don't know whether we have enough money.

  • Do the next 2 steps, Requirements and Priorities, and then do the Budget step. If you cannot afford to do those steps, then you do not have enough money and you should Stop Now.

Requirements

  • The person who know what they want must express what they want in enough detail that the person who is going to provide it can understand what they are supposed to be providing.

This sounds very simple. It is not. Many many conflicts in personal life, disastrous work projects, and famous cockups in politics follow from the fact that helping person A to express what they want well enough so that person B can deliver it for them is a skill that takes time to learn.

Solution: hire a Business Analyst to help person A to express what they want and to express it as requirements that persons A and B can both agree they understand and are happy with

Requirements Checklist

1) Does anyone involved have previous experience of how easy it is is to trip up on getting someone to express what they want correctly and in enough detail for someone to deliver it?

A: No

  • Find someone. Pay for their help.
  • Or, budget for doing everything 2 to 3 times in order to get it right. (Yes, 2 to 3 times is about right for your first ever project. It is not a worst-case.)

Priorities

  • The person who knows what they want must be able to express what their priorities are amongst the things they want. Either based on what they want fastest, or on what they value most.

This sounds very simple. It is. The only two things that can possibly go wrong are:

  • person A changes their mind.
  • you have not broken the work into small enough chunks. A small enough chunk is a chunk that the person doing the work is confident they can can complete in under a week or so.

Priorities Checklist

1) Can the person who knows what they want prioritise what they want and break it down into small enough chunks so that the person who knows how to deliver can confidently say, of at least the top 2 items, that they can do them in less than a week or so each?

A: No

  • Find someone to help with doing this.
  • Alternatively, budget for 2 to 3 times the cost of the first item for working out as you go along how to do it. If you have already budgeted 2 to 3 times cost because of no experience with requirements, that gets you to 4 to 9 times the cost for the first item. Yes, really.

Delivery

  • The person who knows how to deliver (build, buy, borrow, invent, bodge, whatever) what the person who knows what they want wants, does so.
  • [If there is more work involved than they can do in a week or so, they first chunk the work, in strict priority order, into batches of, say, a week's work.]
  • After they have completed as little of the first piece of work that is still showable to the person who knows what they want, they show it. The person who knows what they want confirm or corrects the understanding of what is wanted. The person who is delivering corrects course.
  • After they have completed the whole of the fist piece of work they show that.
  • [Or, in the larger chunked case, after the first batch they show the person who knows what they want, what they can have so far.]
  • The person who knows what they want then says, “yes I'll take that” or “that looks great, keep going” or “please change … <insert detail here>.”
  • If it is not finished, keep going. If it takes a long time, work to a regular rhythm.

Budget

  • After you have requirements and priorities, you may wish to plan a budget. A budget is often counted in money or in person-days of work.

You may think that offering a price for a job is what any experienced tradesman should be able to do. You may, or may not, be right. There are 3 main risks:

  • Nobody involved in the project has enough experience to estimate a fair budget, but no-one wants to admit to not having enough experience.
  • The people estimating the budget are pressured into underestimating the costs
  • The job, for reasons not yet foreeen, will incur unexpected additional costs.

Budget Checklist

1) Make sure you have reviewed each of the 2 to 3 times multipliers for inexperience in Requirements and Prioritisation.

2) Do you have someone on the project with enough experience to estimate a budget?

A: Yes

  • Is that person the person doing the work?
    A: Yes: Then accept their min/max estimates for costs and their choice of a prudent budget
    A: No: Then have them review their min/max estimates for costs with the person doing the work, let the person doing the work provide detail, and let them come to a consensus.

A: No

  • Find someone to help.
  • If you are able and willing to risk the cost of 1 or 2 weeks work, consider simply starting the work and reviewing progress after 1 or 2 weeks. Then, try again to estimate a budget.

Review

  • Whatever you do, review how you are working after 1 to 2 weeks. Ask yourselves questions to help understand your satisfaction and dissatisfaction, for instance:
    • What is going well?
    • What am I happy with or not happy with?
    • What can we do better?
    • What would I like to change?
      Then, make a plan to do at least one of the things you decided would make things better.

Note you are not asking, “is the thing we are delivering good?” You are asking “is the way we are working good?” The two things are closely linked of course, both in the positives and the negatives.

Review Checklist

1) Did you, in fact, do a review after 1 to 2 weeks?
2) Did you, in fact, choose at least one thing to improve?
3) If you are now 4 weeks into a project have you, in fact, improved the things you intended to improve.

Risk

  • A risk is something that might either stop your project or else make you wish you had not started it. Make a list of them.
  • All of the checklists above represent potential risks. If you did not have a whole-hearted pass for each item of each checklist, list that as a risk.
  • Each time you plan a piece of work, consider the associated risks and how you can make the risk smaller. If you don't really know how to, find help.
  • For risks that you cannot make any smaller, are you able to accept the price of that risk happening? Are you clear on who has what share of the price?

Risk Checklist

1) Did you make a risk list?

A: No

  • Stop Now.

2) Do you have someone on the project with prior experience in managing risk?

A: No

  • That is a risk. Get help.

3) Do the things you haved planned to minimise risk really minimise risk or are they just things you can write down even though they might have no effect in minimising the risk?

Considerations

  • It is a fact of craft work that the right answer is often, “Get someone with the experience to do it”
  • Minimalism is of little use to a beginner. Rather, they first need a step-by-step, then later they can understand how to try minimalism.
    This is why complaints that “<insert agile methodology name here> is terrible” are largely futile. All well-known agile methods gives a simple enough way to start off and then require you to review and change how you work.

.Net and Asp.Net on MacOs & Mono instead of Windows (using Visual Studio or JetBrains Rider)

Mono goes a long way in running code written for .Net on Windows. It is all very much easier if you either start with cross-platform in mind, or if you move to .Net Core; but even for existing .Net Framework projects mono can run runs most things including Asp.Net.

Here's my checklist from a couple of years of opening .Net Framework solution files on a Mac and finding they don't build first time.

Most of these require you to edit the .csproj file to make it cross-platform, so a basic grasp of msbuild is very helpful.

  1. For AspNet: inside the PropertyGroup section near the top of the csproj file, add an element:

    <WebProjectOutputDir Condition="$(WebProjectOutputDir) == '' AND $(OS) == 'Unix' ">bin/</WebProjectOutputDir>

    Use this if you get a 'The “KillProcess” task was not given a value for the required parameter “ImagePath” (MSB4044)' error message; or if the build output shows you are trying to create files in an top-level absolute /bin/ path.

  2. For AspNet: Add Condition="'$OS'!='Unix'" to the reference to Microsoft.Web.Infrastructure.dll AND delete the file from the website bin directory.

    <Reference Condition="'$OS'!='Unix'" Include="Microsoft.Web.Infrastructure, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
    <Private>True</Private>
    <HintPath>..\packages\Microsoft.Web.Infrastructure.1.0.0.0\lib\net40\Microsoft.Web.Infrastructure.dll</HintPath>
    </Reference>
  3. For all project types—but, only if you need to use the netCore dotnet build tooling to build an NetFramework project on unix. mono's msbuild does not need this. Add this section somewhere in the csproj file (I put it right at the bottom), to resolve NetFramework4 reference paths:

    <PropertyGroup Condition="$(TargetFramework.StartsWith('net4')) and '$(OS)' == 'Unix'">
    <!-- When compiling .NET SDK 2.0 projects targeting .NET 4.x on Mono using 'dotnet build' you -->
    <!-- have to teach MSBuild where the Mono copy of the reference asssemblies is -->
    <!-- Look in the standard install locations -->
    <BaseFrameworkPathOverrideForMono Condition="'$(BaseFrameworkPathOverrideForMono)' == '' AND EXISTS('/Library/Frameworks/Mono.framework/Versions/Current/lib/mono')">/Library/Frameworks/Mono.framework/Versions/Current/lib/mono</BaseFrameworkPathOverrideForMono>
    <BaseFrameworkPathOverrideForMono Condition="'$(BaseFrameworkPathOverrideForMono)' == '' AND EXISTS('/usr/lib/mono')">/usr/lib/mono</BaseFrameworkPathOverrideForMono>
    <BaseFrameworkPathOverrideForMono Condition="'$(BaseFrameworkPathOverrideForMono)' == '' AND EXISTS('/usr/local/lib/mono')">/usr/local/lib/mono</BaseFrameworkPathOverrideForMono>
    <!-- If we found Mono reference assemblies, then use them -->
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net40'">$(BaseFrameworkPathOverrideForMono)/4.0-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net45'">$(BaseFrameworkPathOverrideForMono)/4.5-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net451'">$(BaseFrameworkPathOverrideForMono)/4.5.1-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net452'">$(BaseFrameworkPathOverrideForMono)/4.5.2-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net46'">$(BaseFrameworkPathOverrideForMono)/4.6-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net461'">$(BaseFrameworkPathOverrideForMono)/4.6.1-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net462'">$(BaseFrameworkPathOverrideForMono)/4.6.2-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net47'">$(BaseFrameworkPathOverrideForMono)/4.7-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net471'">$(BaseFrameworkPathOverrideForMono)/4.7.1-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != '' AND '$(TargetFramework)' == 'net472'">$(BaseFrameworkPathOverrideForMono)/4.7.2-api</FrameworkPathOverride>
    <EnableFrameworkPathOverride Condition="'$(BaseFrameworkPathOverrideForMono)' != ''">true</EnableFrameworkPathOverride>
    <!-- Add the Facades directory.  Not sure how else to do this. Necessary at least for .NET 4.5 -->
    <AssemblySearchPaths Condition="'$(BaseFrameworkPathOverrideForMono)' != ''">$(FrameworkPathOverride)/Facades;$(AssemblySearchPaths)</AssemblySearchPaths>
    </PropertyGroup>
  4. For projects that have lived through C# evolution from C# 5 to C# 7: You may need to remove duplicate references to e.g. System.ValueTuple. Add Condition="'$(OS)' != 'Unix'" to the reference. This applies to Types that MS put on NuGet.org during the evolution. idk why msbuild builds without complain on Windows but not on Unices.
    Example:

    <Reference Condition="'$(OS)' != 'Unix'" Include="System.ValueTuple, Version=4.0.1.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
    <HintPath>..\packages\System.ValueTuple.4.3.1\lib\netstandard1.0\System.ValueTuple.dll</HintPath>
    </Reference>
  5. For References to Microsoft.VisualStudio.TestTools.UnitTesting: Add a nuget reference to MSTEST V2 from nuget.org and make it conditional on the OS

    <ItemGroup Condition="'$(OS)' == 'Unix'">
    <Reference Include="MSTest.TestFramework" Version="2.1.1">
    <HintPath>..\packages\MSTest.TestFramework.2.1.2\lib\net45\Microsoft.VisualStudio.TestPlatform.TestFramework.dll</HintPath>
    </Reference>
    <Reference Include="coverlet.collector" Version="1.3.0" >
    <HintPath>..\packages\coverlet.collector.1.3.0\build\netstandard1.0\coverlet.collector.dll</HintPath>
    </Reference>
    </ItemGroup>

    Note this will only get you to a successful build. To run the tests on unix you then have to download and build https://github.com/microsoft/vstest and run it with e.g.
    mono ~/Source/Repos/vstest/artifacts/Debug/net451/ubuntu.18.04-x64/vstest.console.exe --TestAdapterPath:~/Source/Repos/vstest/test/Microsoft.TestPlatform.Common.UnitTests/bin/Debug/net451/ MyTestUnitTestProjectName.dll.

  6. Case Sensitivity & mis-cased references
    Windows programmers are used to a case-insensitive filesystem. So if code or config contains references to files, you may need to correct mismatched casing. Usually a 'FileNotFoundException' will tell you if you have this problem.

  7. The Registry, and other Permissions
    See this post for more: https://www.cafe-encounter.net/p1510/asp-net-mvc4-net-framework-version-4-5-c-razor-template-for-mono-on-mac-and-linux

Using the command line

It is helpful to be somewhat familiar with microsoft docs on MSBuild Concepts since msbuild will be issuing most of your build errors. If you have installed mono then you can run msbuild from the command line with extra diagnostics e.g.

msbuild -v:d >> build.log

you can also run web applications from the command just by running

xsp

from the project directory.

Original Text from 2011

For reasons best not examined too closely I switch between between Mac and PC which, since I earn my crust largely with .Net development, means switching between Visual Studio and MS.Net and MonoDevelop with Mono.

Mono is very impressive, it is not at all a half hearted effort, and it does some stuff that MS haven't done. But when switching environments, there's always the occasional gotcha. Here are some that have got me, and some solutions.

  • Gotcha: Linq Expressions don't work on Mono?
    Solution: Add a reference to System.Core to your project.
  • Question: What version of NUnit is built in to mono?As of Feb 2011, it's nunit 2.4.8.

Pretending that SQLite has Stored Procedures and Functions

SQLite is marvellous. The fact that it doesn't have SQL syntax for stored procs and functions is usually not a handicap because it has an interface for the consuming application to register functions, which means you get to write your functions in your preferred programming language. Win all round.

But sometimes you do wish you could do it in SQL.

The lack of Stored Procedures is usually ok—you can just use scripts. Variables are easy to do: create a one-row temporary table and call it args or var. Working around the lack of Functions seems harder but in fact, you can program functions with Views. You can use CTEs in a View definition, so you can build up complex calculations. And CTEs allow recursion so you have Turing completeness.

As an example, the Exponent function as a View:

Drop Table if Exists Args ; Create Table Args as Select 5.5 as Base, 4 as Exponent ;

Drop View If Exists Power;
Create View Power As
    WITH RECURSIVE pow(exponent, exponent_remainder, base, result) as (
        --FIRST EXPRESSION
        SELECT exponent, exponent-1 , base, base
        FROM Args

        union all
        --SECOND EXPRESSION
        select Args.exponent, pow.exponent_remainder -1, pow.base, pow.result * pow.base
        from Args
        join pow on Args.exponent = pow.exponent
        where pow.exponent_remainder >= 0
    )
    select pow.result
    from pow
    where pow.exponent_remainder = 0;

and now you ‘call the function’ with:

Update Args set Base=2.5, Exponent=5; Select Result from Power;

The elements of the workaround are:

  1. A one-row table for function arguments
  2. A view which can refer to the arguments table and do the calculation. Since you can use CTEs to do recursion, you could in principle programming anything this way.

In similar style, here's an Exponential function which lets you specify how many significant digits you want the result to, default to about 7 digits of accuracy. This time we call the Args (X,Y,Z,p4,p5,…)

Drop Table if Exists Args ;
Create Table Args as Select 1 as X, 2 as Y, 3 as Z, 4 as p4, 5 as p5, 6 as  p6;

Drop View If Exists Exp;
Create View Exp As
    WITH RECURSIVE exp1(X, N, term, approx, accuracy ) as (
        --FIRST EXPRESSION
        SELECT X, 1, X, 1+X, Max(Min(Y, 1),0.00000000000000001)   FROM Args

        Union All
        --SECOND EXPRESSION
        Select X, N + 1, term * X / (N + 1), approx + term * X / (N + 1), accuracy
        From exp1
        Where  term / approx > accuracy Or N <3
    )
    Select approx as Result From exp1 Order By N Desc Limit 1;

And then:

Update Args Set X=22.0, Y=0.00000000000001;
Select * from Exp;
#
# > 3584912846.1315813 # Exp(22) correct to 14 digits

Links

Installing and using SQLite extensions on macOs (and maybe Windows & Linux too)

Installing and using SQLite extensions on macOs

SQLite is brilliant and … lite. Deliberately. Even for maths functions the view is “it's really, really easy to add extensions and we don't want to bloat the core.”

This is fine if you are used to C development. This page is for if you aren't. The first section is specific to macOs, which is the “hardest” case. Linux and Windows are easier and you can skip the first section.

1. For macOs: Be able to load SQLite extensions

  1. Install SQLite from homebrew, because the apple-shipped SQLite will probably not allow you to load extensions. If you try to load an extension, you will just get a "not authorized" error.
    brew install sqlite

    Note that homebrew tells you that it has not put sqlite in the path because the apple-shipped version is in the path. Fix this either by editing your profile file to extend the path, or else by adding a link to the updated sqlite3 in /usr/local/bin:

    ln -s /usr/local/opt/sqlite/bin/sqlite3 /usr/local/bin/
  2. Now if you start sqlite3 you should see a new improved version number:
    SQLite version 3.33.0 2020-08-14 13:23:32
    Enter ".help" for usage hints.
    

2. Download a Loadable Module

  1. We'll take Spatialite as a great example. Get the .dylib file (macOs) or .dll (for Windows or Linux) or .so file (for Linux) for your extension and confirm it is somewhere you can find it. For Windows the homepage has links to 7z archive file containing the loadable module and sqlite.exe too. For macOs:
    brew install libspatialite
    # ... lots of homebrew output ...
    
    ls /usr/local/lib/*spatialite*
    # /usr/local/lib/libspatialite.dylib
    # /usr/local/lib/mod_spatialite.dylib
    # /usr/local/lib/libspatialite.7.dylib
    # /usr/local/lib/mod_spatialite.7.dylib
    
  2. Add the directory /usr/local/lib to your LD_PATH if it isn't already there. (The alternative to this step is, in the next step, to use the absolute path to load the module.)
    • echo $LD_PATH #Check if you already have it
    • export LD_PATH="$LD_PATH:/usr/local/lib"
    • Edit your profile file to make the change repeatable. For instance:
    • zsh : echo 'export LD_PATH="$LD_PATH:/usr/local/lib"' >> ~/.zshrc
    • bash: echo 'export LD_PATH="$LD_PATH:/usr/local/lib"' >> ~/.bash_profile
    • fish: echo 'set -x LD_PATH "$LD_PATH:/usr/local/lib"' >> ~/.config/fish/config.fish
  3. Start sqlite again and now load the module. There are two ways to do it, either should work:
    • Either with .load :
      .load mod_spatialite.dylib #*if you set LD_PATH above*
      .load /full/path/to/the/file/mod_spatialite.dylib # *if you didn't*
    • Or with Select:

      Select load_extension('mod_spatialite');

    Either way you should now be able to select the spatialite version number:

    select spatialite_version() ;
    # 4.3.0a
    

3. Other Extensions

Spatialite was the easy example because there are pre-compiled binaries available for all platforms. Other extensions mostly exist as .c files. But good news! Many of them are single files and easy to compile and install.

  1. Download some extensions, usually as a single .c file

    For instance, look at https://www.sqlite.org/contrib and notice extension-functions.c at the bottom of the page. Let's install this, which has common maths, string and stats functions functions such as power(), exp(), reverse(), stdev() :

  2. Having downloaded the .c file, compile it in your download directory.

    For macOs:

    gcc -g -fPIC -dynamiclib extension-functions.c -o extension-functions.dylib
    

    For Windows, use one of:

    gcc -g -shared YourCode.c -o YourCode.dll
    cl YourCode.c -link -dll -out:YourCode.dll
    

    For linux/*nix:

    gcc -g -fPIC -shared YourCode.c -o YourCode.so
    
  3. Copy it to your lib directory and use it. e.g. for macOs:
    mv extension-functions.dylib /usr/local/lib/
    sqlite3
    > .load extension-functions.dylib
    Select sqrt(10), log(exp(2)) ;
    # sqrt(10)          log( exp(2) )
    # ----------------  -------------
    # 3.16227766016838  2.0
    

Even more extensions

There are more extensions in the SQLite repository which you can download and install from the .c file in the same way. https://sqlite.org/src/file/ext/misc/ includes files for json & csv formatting, regex, uuids, and other requirements.

If you have a not-latest version of SQLite installed, you may need the advice on Forums - How do I compile the newest .... I ended up with:

for f in *.c
  set b (basename -s .c $f)
  gcc -g -fPIC  -DSQLITE_INNOCUOUS=0 -DSQLITE_VTAB_INNOCUOUS=0  -DSQLITE_VTAB_DIRECTONLY=0  -DSQLITE_SUBTYPE=0 -dynamiclib $f -o $b.dylib
  rm -r $b.dylib.dSYM
end

4. Making it permanent-ish

You can use the file ~/.sqliterc to permanently include your loaded functions. Here's mine:

.headers ON
.mode column
.load extension-functions.dylib

For the same functionality on another machine, you must replicate these steps. The sqlite developers' solution would be, compile your own distribution of sqlite with all the bits you want.

References