Postgres quick start for SQL Server / T-SQL Developers

After 17 years on T-SQL, I at last started working on projects using Postgres. Here are the first things I needed for the transition.

  • Do this first: https://wiki.postgresql.org/wiki/First_steps as it answers first questions about connections, databases and login.
  • Postgres replaces logins and database users and groups with a single concept: ROLE.
    • Permit a ROLE to login with the With Login clause when you Create or Alter it:
      Create Role MyNameHere With Login.
    • Grant it access to a database with Grant Connect: Grant Connect on Database dbname TO roleName.
    • Make it a group with – well, it already is a group. Add other roles to it with Grant Role1 to Role2. I think this approach works out well both for evolving and managing users & groups.
  • No NVarchar is needed, just text. The default installation uses UTF-8. Text is the usual idiom over VarChar(n), allegedly it performs marginally better.
  • 'String concatenation is done with ' || ' a double pipe '.
  • Date & Time: use timestamp for datetime/datetime2. Read Date/Time Types for more on dates, times and intervals.
  • Use semicolon terminators nearly everywhere:
    If ... Then ... ; End If ; T-SQL doesn't need them but Postgres demands them.
  • Before version 11, use Function not Procedure (use returns void if there is no return value). From version 11, you can Create Procedure() but note brackets are needed even for no parameters. A Procedure is called with CALL.
  • A typical definition:
    Create or Replace Function procedureName( forId int, newName varchar(20) )
    Returns int -- use returns void for no return
    As $$
    Declare
      localvariable int;
      othervariable varchar(10);
    Begin
     Insert into mytable (id,name) values (forId, newName) On Conflict (id) Do Update Set Name=NewName ;
    End $$;
    
  • Ooh, did you notice the 'Create or Replace' syntax, and how Postgres has a really good Upsert syntax – Insert On Conflict Update ?
  • Postgres does function overloads, so to drop a function you must give the signature: Drop function functionname(int)
  • Function and Procedural code is strictly separated from “pure” SQL in a way that T-SQL just doesn't bother with, and is typically written in plpgsql script. Declare variables in one block before Begin. For interactive work you can use an Anonymous Function block starting with DO :
    Select 'This line is plain SQL' ;
    Do $$
    Declare
      anumber integer ;
      astring varchar(100);
    Begin
    If 1=1 Then
     Raise Notice 'This block is plpgsql' ;
    End If;
    End $$ ;
    Select 'This line is SQL again' ;
    

    In fact, Postgres functions are defined in a string. The $$ delimiter is not a special syntax for functions, it is the postgres syntax for a literal string constant in which you don't have to escape any special characters at all. So that's handy for multiline strings and quotes.

  • Functions can be defined in other languages than plpgsql. For javascript, for example, see https://github.com/plv8/plv8.
  • If you don't need variables other than parameters, or control statements you can write a routine in pure sql by specifying:
    Create or Replace Procedure ProcName(Date onDate)
    Language Sql
    As $$
    Select * from LatestNews where PublishTimeStamp::Date = onDate
    $$;
    
  • The double colon :: is the cast/conversion operator.
  • Whereas the T-SQLer does everything in T-SQL, other database systems use non-sql commands for common tasks. For working at a command-line, learn about psql meta-commands. Start with \c to change database, \l to list databases, \d to list relations (ie tables), and \? to list other meta commands.
  • Variables: are only available inside plpgsql (or other language) code, not in plain SQL. Just use a plain identifier myvariablename with no @ symbol or other decoration. BUT as a consequence you must avoid variable names in a query that are the same as a column name in the same query. BUT BUT in Ado.Net Commands with Parameters, still use the @parametername syntax as you would for SQL Server
  • Postgres has probably stayed ahead of of T-SQL in keeping close to the SQL standard syntax, though they both become more, not less, standards-compliant with each version. The docs go into detail on deviations. But this means that much of your code for databases, schemas, users, tables, views, etc can be translated fairly quickly.
  • Replace Identity with the SQL Standard GENERATED BY DEFAULT AS IDENTITY for Postgres 10 onwards. (You will see Serial in older versions). For more complex options, Postgres uses SQL Sequences.
  • The Nuget package for Ado.Net, including for .Net core, is Npgsql
  • Scan these

Converting T-SQL to Postgres

Here's my initial search/replace list for converting existing code:

Search Replace
@variableName _VariableName
but stay with @variable for .Net clients using Npgsql
NVarChar Text
datetime
datetime2
timestamp
Identity Generated By Default As Identity
Before postgres 10 use Serial
Raise Raise Exception
Print Raise Notice
Select Top 10 ... Select ... Fetch First 10 Rows Only ;
Insert Into table Select ... Insert into Table (COLNAMES) Select ...
If Then Begin … End
Else Begin … End
If Then … ;
Else … ; End If ;
Create Table #Name Create Temporary Table _Name
IsNull Coalesce
UniqueIdentifier uuid
NewId() Before version 13, first websearch for Create Extension "uuid-ossp" to install the guid extension to a database. Then you can use uuid_generate_v4() and other uuid functions
Alter Function|Procedure Create or Replace Function

Authorization

Usually you will connect to postgres specifying a username. The psql.exe commandline tool will default it to your OS login username. You can avoid passwords in scripts that use psql by putting them in the pgpass.conf file).

If you want to set up integrated security, the limitation is that for computers not in a Domain it only works on localhost. The instructions at Postgres using Integrated Security on Windows take about 5 minutes for the localhost case, and include a link to the extra steps for the Domain case.

GUI Admin

Your replacement for SQL Server Manager is the pgAdmin GUI which gets you nicely off the ground with a live monitoring dashboard.

Command Line and Scripting

psql -h localhost -d myDB -c 'Select Current_User'

runs the quoted command. Note -h for “host”, not -s for “server”. Use psql --help to see more options. You can also use the pipe:

echo 'Select Version(), Current_Database();
Select Current_TimeStamp' | psql -h localhost

For large data dumps, pg_dump and pg_restore are designed to generate consistent backups of entire databases or selected tables —data or DDL or both—without blocking other users.

There are more command line utilities

Conway’s Law & Distributed Working. Some Comments & Experience

The eye-opener in my personal experience of Conway's law was this:

A company with an IT department on the 1st floor, and a marketing department on the 2nd floor, where the web servers were managed by the marketing department (really), and the back end by the IT department.

I was a developer in the marketing department. I could discuss and change web tier code in minutes. To get a change made to the back end would take me days of negotiation, explanation and release co-ordination.

Guess where I put most of my code?

Inevitably the architecture of the system became Webtier vs Backend. And inevitably, I put code on the webserver which, had we been organised differently, I would have put in a different place.

This is Conway's law: That the communication structure – the low cost of working within my department vs the much higher cost of working across a department boundary – constrained my arrangement of code, and hence the structure of the system. The team "just downstairs" was just too far.  What was that gap made of? Even that small physical gap raised the cost of communication; but also the gaps & differences in priorities, release schedules, code ownership, and—perhaps most of all—personal acquaintance; I just didn't know the people, or know who to ask.

Conway's Law vs Distributed Working

Mark Seemann has recently argued that successful, globally distributed, OSS projects demonstrate that co-location isn't all it's claimed to be. Which set me thinking about communication in OSS projects.

In my example above, I had no ownership (for instance, no commit rights) to back end code and I didn't know, and hence didn't communicate with, the people who did. The tools of OSS—a shared visible repository, the ability to 'see' who is working on what, public visibility of discussion threads, being able to get in touch, to to raise pull requests—all serve to reduce the cost of communication.

In other words, the technology helps to re-create, at a distance, the benefits enjoyed by co-located workers.

When thinking of communication & co-location, I naturally think of talking. But @ploeh's comments have prodded me into thinking that code ownership is just as big a deal as talking. It's just something that we take for granted in a co-located team. I mean, if your co-located team didn't have access to each other's code, what would be the point of co-locating?

Another big deal with co-location is "tacit" knowledge, facilitated by, as Alistair Cockburn put it, osmotic communication. When two of my colleagues discuss something, I can overhear it and be aware of what's going on without having to be explicitly invited. What's more, I can quickly filter out what isn't relevant to me, or I can spontaneously join conversations & decisions that do concern me. Without even trying, everyone is involved when they need to be in a way that someone working in a separate room–even one that's right next door–can't achieve.

But a distributed project can achieve this too. By forcing most communication through shared public channels—mailing lists, chatrooms, pull request conversations—a distributed team can achieve better osmotic communication than a team which has two adjacent rooms in a building.

The cost, I guess, is that typing & reading is more expensive (in time) than talking & listening. Then again, the time-cost of talking can be quite high too (though not nearly as a high as the cost of failing to communicate).

I still suspect that twenty people in a room can work faster than twenty people across the globe. But the communication pathways of a distributed team can be less constrained than those same people in one building but separated even by a flimsy partition wall.

References

IIS Express : Run a child web application in a virtual directory under a parent application

Like this: Edit your IIS Express config file at [shell]"%userprofile%\My Documents\IISExpress\config\applicationhost.config"[/shell]
Create a site which has two applications defined in it, e.g.
[xml]<site name="MyTopLevelAndChildWebAppsInOneSite" id="123" >
<application path="/" applicationPool="Clr4IntegratedAppPool">
<virtualDirectory path="/" physicalPath="C:\Users\me\Source\TopLevelWebApp" />
</application>
<application path="/Child" applicationPool="Clr4IntegratedAppPool">
<virtualDirectory path="/" physicalPath="C:\Users\me\Source\ChildWebApp" />
</application>
<bindings>
<binding protocol="http" bindingInformation="*:51234:localhost" />
</bindings>
</site>[/xml]
And then run the site, matching it on the siteid:
[shell]start "Woo!" "C:\Program Files (x86)\IIS Express\iisexpress.exe" /siteid:123[/shell]

Browse to, and close, your web apps in the usual way from the IIS Express icon in the systray.

Optionally, experience the pain that is web.config inheritance. But try not to.

An Asp.Net MVC HtmlHelper.RadioButtonsFor helper

These overloads will do the hopefully-obvious thing with a Model => Model.Property and a List of SelectListItems (or a single SelectListItem)

Use the RadioButtonLabelLayout setting to control whether you nest the radio button inside its label, or lay them out as siblings; and whether you like text before button or vice versa.

[code language="csharp"]
public static class HtmlHelperRadioButtonExtensions
{
/// <summary>
/// Returns radio buttons for the property in the object represented by the specified expression.
/// A radio button is rendered for each item in <paramref name="listOfValues"/>.
/// Use <paramref name="labelLayout"/> to control whether each label contains its button, or is a sibling,
/// and whether button precedes text or vice versa.
/// <list type="bullet">
/// <item>
/// <term>Example result for the default labelLayout= RadioButtonLabelLayout.LabelTagContainsButtonThenText:</term>
/// <description>
/// &lt;label for="Object_Property_Red"&gt;&lt;input id="Object_Property_Red" name="Object.Property" type="radio" value="Red" /&gt; Red&lt;/label&gt;
/// &lt;label for="Object_Property_Blue"&gt;&lt;input id="Object_Property_Blue" name="Object.Property" type="radio" value="Blue" /&gt; Blue&lt;/label&gt;
/// </description>
/// </item>
/// <item>
/// <term>Example result for labelLayout= RadioButtonLabelLayout.SiblingBeforeButton:</term>
/// <description>
/// &lt;label for="Object_Property_Red"&gt;Red&lt;/label&gt; &lt;input id="Object_Property_Red" name="Object.Property" type="radio" value="Red" /&gt;
/// &lt;label for="Object_Property_Blue"&gt; Blue&lt;/label&gt; &lt;input id="Object_Property_Blue" name="Object.Property" type="radio" value="Blue" /&gt;
/// </description>
/// </item>
/// </list>
/// </summary>
/// <param name="listOfValues">Used to generate the Value and the Label Text for each radio button</param>
/// <param name="labelLayout">One <see cref="RadioButtonLabelLayout"/> to control the layout of the rendered button and its label.</param>
/// <returns>
/// An MvcHtmlString for the required buttons
///
/// </returns>
public static MvcHtmlString
RadioButtonsFor<TModel, TProperty>(this HtmlHelper<TModel> htmlHelper,
Expression<Func<TModel, TProperty>> expression,
IEnumerable<SelectListItem> listOfValues,
RadioButtonLabelLayout labelLayout = RadioButtonLabelLayout.LabelTagContainsButtonThenText)
{
if (listOfValues == null) { return null; }
var buttons= listOfValues.Select(item => RadioButtonFor(htmlHelper, expression, item, labelLayout));
return MvcHtmlString.Create(buttons.Aggregate(new StringBuilder(), (sb, o) => sb.Append(o), sb => sb.ToString()));
}

/// <summary> Create an <see cref="IEnumerable{T}"/> list of radio buttons ready for individual processing before rendering</summary>
public static IEnumerable<MvcHtmlString> RadioButtonListFor<TModel, TProperty>(
this HtmlHelper<TModel> htmlHelper,
Expression<Func<TModel, TProperty>> expression,
IEnumerable<SelectListItem> listOfValues,
RadioButtonLabelLayout labelLayout)
{
if (listOfValues == null) { return new MvcHtmlString[0]; }
return listOfValues.Select( item => RadioButtonFor(htmlHelper, expression, item, labelLayout) );
}

public static MvcHtmlString RadioButtonFor<TModel, TProperty>(
HtmlHelper<TModel> htmlHelper,
Expression<Func<TModel, TProperty>> expression,
SelectListItem item,
RadioButtonLabelLayout labelLayout)
{
var id = htmlHelper.IdFor(expression) + " " + item.Value;
var radio = htmlHelper.RadioButtonFor(expression, item.Value, new {id}).ToHtmlString();
var labelText = HttpUtility.HtmlEncode(item.Text);
TagBuilder nestingLabel = null;
switch (labelLayout)
{
case RadioButtonLabelLayout.LabelTagContainsTextThenButton:
nestingLabel = new TagBuilder("label") {InnerHtml = labelText + " " + radio};
return MvcHtmlString.Create(nestingLabel.ToString());
case RadioButtonLabelLayout.LabelTagContainsButtonThenText:
nestingLabel = new TagBuilder("label") {InnerHtml = radio + " " + labelText};
return MvcHtmlString.Create(nestingLabel.ToString());
case RadioButtonLabelLayout.SiblingBeforeButton:
return MvcHtmlString.Create(radio + " " + htmlHelper.Label(id, labelText));
case RadioButtonLabelLayout.SiblingAfterButton:
return MvcHtmlString.Create(htmlHelper.Label(id, labelText) + " " + radio);
default:
throw new ArgumentOutOfRangeException("labelLayout", labelLayout,
"This is not a valid RadioButtonLayoutStyle for rendering a radio button");
}
}
}

public enum RadioButtonLabelLayout
{
LabelTagContainsButtonThenText = 0,
LabelTagContainsTextThenButton = 1,
SiblingBeforeButton = 2,
SiblingAfterButton = 3,
}[/code]