Building SQL Server Database Project In Ubuntu

6uxekuva 于 2023-08-02 发布在 SQL Server

关注(0)|答案(7)|浏览(142)

I'm building an ASP.NET Core 2.0 Web API application that is hosted in an Ubuntu environment. So far, I've had great success getting things building and running (for the .NET Core app) in Ubuntu.

For the database, I have a SqlProj included in my solution. The project includes typical things such as tables, SPs, and pre/post deployment scripts. I'm using the following command (on my Windows-based dev machine) to build and deploy this project:

msbuild .\MyProject.DB.sqlproj /t:Build /t:Publish /P:SqlPublishProfilePath="./PublishProfiles/MyProject.DB.publish.xml"

When I take this approach, everything builds and deploys properly; however, since I will be taking advantage of the .NET Core CLI commands + CI/CD that targets an Ubuntu environment, I'd like to do something more like:

dotnet msbuild .\MyProject.DB.sqlproj /t:Build /t:Publish /P:SqlPublishProfilePath="./PublishProfiles/MyProject.DB.publish.xml"

In Windows, I immediately get the error:

error MSB4019: The imported project "C:\Program Files\dotnet\sdk\2.1.4\Microsoft\VisualStudio\v11.0\SSDT\Microsoft.Data.Tools.Schema.SqlTasks.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

Basically, what I'm asking is how to successfully build and deploy a SqlProj project in an Ubuntu environment. I've tried Googling, but I have had zero luck thus far. All of the similar issues that I've found were for individuals who were editing their .proj file to target their VS folder's SSDT. All of these individuals were fixing the issue in Windows. This approach will not work in Ubuntu, since the targets file uses Windows registry keys.

EDIT: I'm aware that SSDT is needed in order to perform such a deployment using MSBuild. I've found no evidence that installing/using SSDT is even possible in Ubuntu. If it is not, perhaps there is an alternative solution?

FYI, I'm aware that using a code-first approach with EF Core is possible. I'm attempting to take the raw SP approach (along with leveraging indexes) and keep track of all of my code using SqlProj instead. This will all be stored and CI/CDed from a Git repo.

sql-server

来源：https://stackoverflow.com/questions/48481028/building-sql-server-database-project-in-ubuntu

7条答案

按热度按时间

kiz8lqtg1#

You can use this NuGet package to deploy without installing SSDT https://www.nuget.org/packages/Microsoft.Data.Tools.Msbuild I don't know if it will run on Ubuntu or integrate at all with the dotnet cli

赞(0）回复(0）举报 2023-08-02

eufgjt7s2#

My 2020 Solution

I would like to revisit this in 2020 with an updated answer to my original question.

I have taken a different approach to building an deploying SQL Server projects. My current approach is to build a pipeline that uses a vs2017-win2016 agent and use this to build a .dacpac. From there, you build a deployment pipeline to deploy the dacpac (from your artifact drop) out to the SQL Server instance.

This approach better accommodates DevOps methodologies and removes the manual process associated with my previous solution.

You can read more about this here:

https://learn.microsoft.com/en-us/azure/devops/pipelines/apps/aspnet/build-aspnet-dacpac?view=azure-devops

赞(0）回复(0）举报 2023-08-02

sqougxex3#

As an alternative, it is possible to achieve this with dotnet cli and sqlpackage as explained here using an MSBuild Sdk.

You basically have a database project. Let's call it "DatabaseProject".

You create a new project which is a .NET standard c# library that you can call "DatabaseProject.Build".

Then you can configure you DatabaseProject.Build.csproj as such:

<Project Sdk="MSBuild.Sdk.SqlProj/1.11.4">
   <PropertyGroup>
       <TargetFramework>netstandard2.0</TargetFramework>
       <Configurations>Debug;Release</Configurations>
   </PropertyGroup>
   <ItemGroup>
       <Content Include="..\DatabaseProject\**\*.sql" />
       <Content Remove="..\DatabaseProject\bin\*.sql" />
       <Content Remove="..\DatabaseProject\**\*.PostDeployment.sql" />
       <PostDeploy Include="..\DatabaseProject\**\*.PostDeployment.sql" />
   </ItemGroup>
</Project>

Be Aware The version used V1.11.4 is the one that supports the current .NET SDK shipped with visual studio at the time of the edit of this post. Check out the github repo to get the latest nuget version for your projet.

Using dotnet build will generate a dacpac that you will be able to use with either dotnet publish or sqlpackage .

You can then publish to you SqlServer instance. If you're like me using a linux runner in your CI, you'll probably need SqlServer authentification method and then run either

sqlpackage /Action:Publish \    
/SourceFile:\"DatabaseProject.Build/bin/Debug/netstandard2.0/DatabaseProject.Build.dacpac\" \
/TargetServerName:MyDatabaseServerName \
/TargetDatabaseName:MyDatabaseName \
/TargetUser:Username\
/TargetPassword:Password

or using a profile generated by visual studio :

sqlpackage /Action:Publish /Profile:\"DatabaseProject/PublishProfile/MyProfile.publish.xml\" /SourceFile:\"DatabaseProject.Build/bin/Debug/netstandard2.0/DatabaseProject.Build.dacpac\"

dotnet publish /p:TargetServerName=MyServerName /p:TargetDatabaseName=MyDatabseName /p:TargetUser=<username> /p:TargetPassword=<password>

赞(0）回复(0）举报 2023-08-02

dm7nw8vv4#

Azure Data Studio now has an extension that lets you build database projects (sqlproj) using the dotnet tool. The brains behind building the project lies in the SQL Server Tools package, which is where the extension gets the required "BuildDirectory" DLL and targets dependencies.

Though not documented, if you want to set this up completely headless outside of Azure Data Studio, you can follow their CLI guide, https://learn.microsoft.com/en-us/sql/azure-data-studio/extensions/sql-database-project-extension-build-from-command-line?view=sql-server-ver15 , but instead extract the necessary files from the RHEL release in https://github.com/microsoft/sqltoolsservice/releases and then follow the rest of the extension's documentation. Here is a working Dockerfile that demonstrates the approach:

FROM mcr.microsoft.com/dotnet/sdk:6.0

WORKDIR /app

RUN apt-get update \
    && apt-get install -y curl

# SSDT dlls and targets file used by Azure Data Studio Extension can be found in the SQL Tools Service project
RUN curl -sSL -o /tmp/sqltools.tar.gz https://github.com/microsoft/sqltoolsservice/releases/download/v3.0.0-release.181/Microsoft.SqlTools.ServiceLayer-rhel-x64-net6.0.tar.gz

# Extract files that are required per https://learn.microsoft.com/en-us/sql/azure-data-studio/extensions/sql-database-project-extension-build-from-command-line?view=sql-server-ver15

RUN mkdir /tmp/sqltools && tar -xzf /tmp/sqltools.tar.gz -C /tmp/sqltools && \
  mkdir /app/BuildDirectory && cd /tmp/sqltools && cp \
    Microsoft.Data.SqlClient.dll \
    Microsoft.Data.Tools.Schema.Sql.dll \
    Microsoft.Data.Tools.Schema.SqlTasks.targets \
    Microsoft.Data.Tools.Schema.Tasks.Sql.dll \
    Microsoft.Data.Tools.Utilities.dll \
    Microsoft.SqlServer.Dac.dll \
    Microsoft.SqlServer.Dac.Extensions.dll \
    Microsoft.SqlServer.TransactSql.ScriptDom.dll \
    Microsoft.SqlServer.Types.dll \
    System.ComponentModel.Composition.dll \
    System.IO.Packaging.dll \
  /app/BuildDirectory && \
  rm -r /tmp/sqltools

#dotnet build your-database-project.sqlproj /p:NetCoreBuild=true /p:NETCoreTargetsPath="/app/BuildDirectory"

The commented command at the end shows what you could run inside the container in the directory with your database project.

This can also then be combined with a container utilizing sqlpackage to implement a full dacpac build and publish automation toolset.

赞(0）回复(0）举报 2023-08-02

b5lpy0ml5#

I can't speak to whether or not this will work on Ubuntu, but we recently got through this on a Windows build machine that does not have SSDT installed, using the NuGet package mentioned above. The breakthrough came from piecing together the details in the article below, specifically that using the SDK with MSBuild needed to have environment variables set in order to work.

https://blogs.msdn.microsoft.com/ssdt/2016/08/22/part-5-use-your-own-build-and-deployment-agent/

With that added info, we installed the NuGet package in the root of the solution folder and then wrote a build script in PowerShell. The script sets the environment variables first and then calls MSBuild on the SqlProj file with the appropriate output directory. We don't specifically publish at that point, but instead publish the artifact to Octopus Deploy in our workflow which does the actual deployment.

Again, not sure it will help on Ubuntu, but thought the additional detail might be useful.

赞(0）回复(0）举报 2023-08-02

6bc51xsx6#

As mentioned, the easiest way to build DacPac file on a linux agent is done via MSBuild.Sdk.SqlProj Go to your database project directory in parallel to .sqlproj file create a directory like DB.Build under it create DB.Build.csproj copy.pase the content as below

<Project Sdk="MSBuild.Sdk.SqlProj/1.1.0"> <!-- This will pull in the required tools and dependencies to build a .dacpac with .NET Core -->
    <PropertyGroup>
        <TargetFramework>netstandard2.0</TargetFramework>
    </PropertyGroup>
    <ItemGroup>
        <Content Include="..\src\DB\masterdata\**\*.sql" /> <!-- link in the new .csproj to the .sql scripts in your existing database project -->
    </ItemGroup>
</Project>

After run you will see dacpac file appears under DB.Build/bin/Release/netstandard2.0/DB.Build.dacpac

Here's my build agent output (Ubuntu agent on Azure devops)

Starting: SQL DB build Release
==============================================================================
Task         : .NET Core
Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version      : 2.187.0
Author       : Microsoft Corporation
Help         : https://learn.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================
Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
/opt/hostedtoolcache/dotnet/dotnet build /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/RecommenderAPI.DB.Build.csproj -dl:CentralLogger,"/home/vsts/work/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.187.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll"*ForwardingLogger,"/home/vsts/work/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.187.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll" --configuration Release /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation=/home/vsts/work/1/recommender-service-cicd/DacPac/
Microsoft (R) Build Engine version 16.5.0+d4cbfca49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 51.72 ms for /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/RecommenderAPI.DB.Build.csproj.
  Using package name RecommenderAPI.DB.Build and version 1.0.0
  Using SQL Server version Sql150
  Deleting existing file /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/obj/Release/netstandard2.0/RecommenderAPI.DB.Build.dacpac
  Writing model to /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/obj/Release/netstandard2.0/RecommenderAPI.DB.Build.dacpac
  RecommenderAPI.DB.Build -> /home/vsts/work/1/s/src/RecommenderAPI.DB/RecommenderAPI.DB/RecommenderAPI.DB.Build/bin/Release/netstandard2.0/RecommenderAPI.DB.Build.dacpac

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.71

Finishing: SQL DB build Release

Note: Make sure to restore you NuGet packages in step prior to build

赞(0）回复(0）举报 2023-08-02

enyaitl37#

The package often suggested by others, R&R WFM's MSBuild.Sdk.SqlProj does not come from Microsoft, and it doesn't really do a "real SSDT build". It does not use SSDT build tools, it is a separate 3rd party tool that can generate limited DACPAC packages with reduced features. This info does not come from my experience with that tool. I read it somewhere when I tried to research what options I have left as I'm stuck with DACPACs and Linux. While I can't confirm that info in any way, it looks true, repo doesn't have 'Microsoft' logos anywhere, and I read such warnings on multiple unrelated sites, posted by supposedly different people.

That being said, it's an awesome tool that certainly required huge amount of workand knowledge.

But there's another one as well.

Recently Microsoft started developing their own cross-platform SSDT build tool, Microsoft.Build.Sql coming from DacFx . At the time of writing this, it's still in preview and the current version is 0.1.10 . You'll often see 0.1.3 or 0.1.7 if you search for articles about how to use that.

Microsoft provided a nice guide how to convert an existing .sqlproj generated by VisualStudio+SSDT. It's concise, a bit too concise, and requires minor obvious fixes. The article also points to a minimalistic new-style .sqlproj file, the link is broken, but the file resides in a nearby folder .

The article mentions that new-style projects support globbing (so Include="folder/**/*.sql" instead of listing all files), and that including all SQL files is already the default behavior, so an 'empty' project file like the sample one will grab all SQL files - but be careful, and read the final paragraph about compatibility with VisualStudio.

Using this info, I was able to convert my .sqlproj file. I did the steps explained there, and also I didn't remove any now-not-needed file inclusions, so VS is happy.

My .sqlproj file looks now like that:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0">
  <!-- IMPORTANT: this is a PREVIEW tooling, keep an eye on changes and notes -->
  <!-- https://github.com/microsoft/DacFx/blob/main/src/Microsoft.Build.Sql/docs/Converting-Existing.md -->
  <Sdk Name="Microsoft.Build.Sql" Version="0.1.10-preview" />
  <PropertyGroup>
    <Name>xxxxxxxxxxxxxxxx</Name>
    <DSP>Microsoft.Data.Tools.Schema.Sql.Sql130DatabaseSchemaProvider</DSP>
    <ModelCollation>1033,CI</ModelCollation>
    <TargetDatabaseSet>True</TargetDatabaseSet>
    <DefaultCollation>Latin1_General_100_CI_AS</DefaultCollation>
  </PropertyGroup>
  <ItemGroup>
    <Folder Include="Properties" />
    <Folder Include="Tables" />
    <Folder Include="Views" />
    <Folder Include="Procedures" />
  </ItemGroup>
  <ItemGroup>
    <Build Include="Tables\xxxxxxxxx.sql" />
    <Build Include="Tables\xxxxxxxxx.sql" />
    <Build Include="Views\xxxxxxxxx.sql" />
    <Build Include="Views\xxxxxxxxx.sql" />
    <Build Include="Procedures\xxxxxxxxx.sql" />
    <Build Include="Procedures\xxxxxxxxx.sql" />
  </ItemGroup>
  <ItemGroup>
    <RefactorLog Include="xxxxxxxxx.refactorlog" />
  </ItemGroup>
  <ItemGroup>
    <PostDeploy Include="Script.PostDeployment1.sql" />
  </ItemGroup>
  <ItemGroup>
    <PreDeploy Include="Script.PreDeployment1.sql" />
  </ItemGroup>
  <ItemGroup>
    <None Include="xxxxxxxxx.publish.xml" />
  </ItemGroup>
</Project>

That 'xxx' parts is just anonymization of course.

I was able to successfully build this project and get a DACPAC file by simple

cd solutionroot
dotnet build MyProject.sln

cd solutionroot/dbproject
dotnet build MyProject.sqlproj

I did not have to add /p:NetCoreBuild=true - the tooling in 0.1.10 detects that it is ran from dotnet command and sets this automatically.

This worked both when the build was done at my local dev machine (Win10, VS2022, dotnet --version = 7.0.302), and when ran on CI agent (Amazon Linux 2).

The CI system was AWS CodeBuild and script used was:

version: 0.2    
phases:
  install:
    runtime-versions:
      dotnet: latest

  pre_build:
    commands:
      - dotnet --version
      - dotnet restore

  build:
    commands:
      - dotnet build -c Release

Not much happening really. The relevant output parts were:

[Container] 2023/07/08 20:36:06 Running command dotnet --version
6.0.408
(...snip...)
  xxxx.Database -> /codebuild/output/src3703237027/src/git-codecommit.eu-west-3.amazonaws.com/v1/repos/xxxx/xxxx/xxxx/bin/Release/xxxx.dll
  xxxx.Database -> /codebuild/output/src3703237027/src/git-codecommit.eu-west-3.amazonaws.com/v1/repos/xxxx/xxxx/xxxx/bin/Release/xxxx.dacpac

So it actually took 6.0, different than on my local. A bit odd, but it still built the dacpac.

Two things worth noting - while dotnet build is able to build a DACPAC, the tooling is "preview" and if I remember correcly, the docs at GitHub mention that not all features are supported. I have no idea how the feature gaps compare to the gaps in rr-wfm's nuget, but the tool is still developed, so there's hope :)

The second thing to note is that while dotnet build is able to build the DACPAC, the dotnet publish is NOT able to run it. Trying to

dotnet publish -c Release -p:SqlPublishProfilePath=./xxxx.publish.xml

at best currently ends up with following error

/root/.nuget/packages/microsoft.build.sql/0.1.10-preview/tools/netstandard2.1/Microsoft.Data.Tools.Schema.SqlTasks.targets(1869,7): error MSB4018: The "SqlPublishTask" task failed unexpectedly.
/root/.nuget/packages/microsoft.build.sql/0.1.10-preview/tools/netstandard2.1/Microsoft.Data.Tools.Schema.SqlTasks.targets(1869,7): error MSB4018: System.PlatformNotSupportedException: Microsoft.Data.SqlClient is not supported on this platform.
/root/.nuget/packages/microsoft.build.sql/0.1.10-preview/tools/netstandard2.1/Microsoft.Data.Tools.Schema.SqlTasks.targets(1869,7): error MSB4018:    at Microsoft.Data.SqlClient.SqlConnectionStringBuilder..ctor(String connectionString)
/root/.nuget/packages/microsoft.build.sql/0.1.10-preview/tools/netstandard2.1/Microsoft.Data.Tools.Schema.SqlTasks.targets(1869,7): error MSB4018:    at Microsoft.Data.Tools.Schema.Common.SqlClient.SqlConnectionFactory..ctor(String connectionString)
/root/.nuget/packages/microsoft.build.sql/0.1.10-preview/tools/netstandard2.1/Microsoft.Data.Tools.Schema.SqlTasks.targets(1869,7): error MSB4018:    at Microsoft.Data.Tools.Schema.Sql.Deployment.CachedRegistryConnectionString..ctor(String connectionString, Boolean loadFromRegistry)

I get this error BOTH when I try to run the dotnet publish on my local dev machine (where everything worked), and obviously also when I try that at the CI agent.

This is because the dotnet publish command isn't currently supported by the 0.1.10 tooling. It probably will not be supported at all, for some probably important, probably internal reason.

Instead of dotnet publish , a separate tool is provided by DacFx, called SqlPackage . It's advertised as the cross-platform tool for running DACPACs.

It can be installed with dotnet :

dotnet tool install -g microsoft.sqlpackage

and it can publish the DACPAC, although with a bit different commandline syntax (see the previous link above for details):

SqlPackage /Action:Publish /SourceFile:".dacpac" /Profile:"....."

I have not tried installing and running this SqlPackage tool, but looks very promising.

赞(0）回复(0）举报 2023-08-02

我来回答

Building SQL Server Database Project In Ubuntu

7条答案

My 2020 Solution

相关问题

热门标签

最新问答