Tag Archives: inventory

Simple Database Inventory Manager 2.3

We got a job.
We got a job.

Always moving forward, here at KnowledgeHunter Corp! (Note: not a real corporation) Just posted the github of the new SDIM 2.3. Changes in just about every corner, but the big overall change is the addition of job information in the data pull.

If you have no idea what SDIMS is or I’m yammering about, here’s the run-down. Otherwise, read on for the 2.3 update news.

Dey took er jerbs
New buttons in 2.3 include 'Jobs' and 'Jobs Extended'
New buttons in 2.3 include ‘Jobs’ and ‘Jobs Extended’

The newest feature is ability to inventory pretty thorough job information. Well, the stuff I’m always looking for anyway. Also included is some extended information, which right now basically includes just a description with a few other identifying columns. Future versions may gather more data and put it here. Powershell handles it well, even with several thousand rows/jobs.

Here is the kind of information that you can expect to see:

  • Server/Instance
  • Job Name
  • Is the Job enabled?
  • Schedule Name
  • Is the Schedule enabled?
  • Schedule Type, Occurrence, Recurrence (if applicable) and Frequency
  • Job Description

This information has been added to the ‘Full Inventory’, so be aware that has a lot of new stuff in it too.

Full disclosure: I used heavily modified version of a few queries from Dattatrey Sindol to pull jobs data.

Always Squashing

Fixed a few bugs:

isProduction column now does not display in Instances list. That was a feature that got removed from everywhere (before 1.0). Finally removed the column.

HIPAA level feature is now completely removed. Never got this one working right and I decided that in the future I’ll go with something a bit more general, like maybe just ‘priority’ or something.

Fixed a few typos. Me spel gud now.

Known Issues:

Too many varchar(max) columns. I know. I know.

Me Want

So, how do you get this? Well…

If you never have used it before, I would suggest going through the walkthrough I put out a while ago. I may in the future build a more simplified version for people who just want to install and don’t care about the behind-the-scenes. (UPDATE: I did just that, here)

Elseif you’re using prior to 2.1 already, then you should probably just drop all the SQL objects and rebuild with the new 2.3 items. This includes the PowerShell datapull and frontend pieces; they’re the only two PowerShell pieces.

Elseif you’re on 2.1 right now you can just replace the items that were changed. I have created a script that will do this for you, because that’s the kind of guy I am. You may need to run the script twice because I didn’t check for object existence. Running it multiple times won’t hurt. It doesn’t touch the Servers table, but it does drop the InstanceList, so if you have that statically assigned, then you’re going to want to back it up and then re-insert the data with the missing columns removed.

And Here’s A Count of All the Bolts

Lastly, here’s the items that were altered in the 2.1 – 2.3 release if you’re curious:

Tables:
dbo.JobList – This holds all the jobs information. Added for new feature (look! No varchar(max)!)
dbo.InstanceList – Removed a column.

Stored Procedures:
dbo.prGetInstances – Altered to fix bug
dbo.prGetInventory – Altered to fix bug
dbo.prGetJobs – New for new feature
dbo.prGetJobsExt – New for new feature
dbo.prGetServersAndInstances – Altered to fix bug
dbo.prInsertJobList – New for new feature
dbo.prUpdateInstanceList – Altered to fix bug

PowerShell:

DB_DataPull – Altered to pull Jobs (new feature) and fix bugs
DB_DataPUll_FrontEnd – Altered to display Jobs/Jobs Extended

Simple Database Inventory Manager 2.1

sdim_ver_2-1_trunc_275x400

A while back, I took on a fairly big project; build an database inventory manager that did the following:

  • Dynamically gathered information on SQL Instances/Databases
  • Dynamically gathered information on the OS underneath
  • Compiled and organized this data in a single repository
  • Provided a client GUI front-end for ease of use.
  • Was built on free and already available tools.

When I initially finished Inventory Manger 1.0 I wrote a 5-part series that took the user through the steps of how I built what I did and how everything worked. This was a good way for me to iron out details and also provide some documentation along the way. As the months have progressed I added updates to the code and altered the posts as necessary.

Then everything went silent as I moved on to other things, but I was constantly going back and adding new features and options. During this interim, I was not able to get the code updated on GitHub, and thus it fell behind. Also, it had morphed into its own beast, moving out of “project” and more into a standard “software” mode. To signify its new direction, its name is now Simple Database Inventory Manager™.

As such, I have added the new 2.1 version to its own GitHub project (previously it existed in the SQL code section there with other unrelated snippets) and will update it accordingly. You can just download the new files and overwrite the old ones to get the new version. Spiffy, right?

db_datapull_frontend_v2-1
Look! 200% more buttons!™

I will keep the old 1.x version in the original repository so it doesn’t break links for the walk-through articles.

All that said, what fabulous prizes are in the new version? Glad you asked!

CMS Integration

Now you can point to your Central Management Server and the Simple Database Inventory Manager™ will pull a list of servers and instances from there. It will then go through all of them recursively, and pull whatever data you want back.

In DB_DataPull.ps1, you can switch this on or off with the -UseCMS switch. If on, then you will need to specify the CMS Server with the -CMSServer ‘SOMESERVER’ parameter.

If you use the -UseCMS option, this will delete all data from the repository tables and repopulate them based on what you have in the CMS. This is backwardly compatible with the old system in that if you don’t use the option (off by default) then it will continue to use the Server and Instance List you provided manually.

Fresh, New Buttons

‘Services’ has been folded into the Inventory heading and Other Info is no more, replaced by the Reporting section (below).

Composite buttons have been created to give better side-by-side information from the GUI. Server\Instance and Instance\DB buttons have been added to do this. I think their names are pretty self-explanatory. Click on them. See what happens.

Reporting

Kind of. A bunch of stock reports were added to the DB_DataPull_FrontEnd.ps1 under the Reporting heading. These rely on Views in the Reporting Schema that build customizable information you want returned.

I’m going to leave this schema (Reporting) pretty much alone for now, so users can create their own views and then use those as datasets if you want for SSRS. You can use SDIM™ as the basis for reports that refresh as often as you want to run DB_DataPull.ps1

Here’s a quick list of the out-of-the-box reports given in the GUI:

Servers Grouped by OS and Service Pack – A count of all Server OS names and versions.

Instances Grouped By SQL Version – A count of all Instances at a specific server version (ignores Service Pack)

Instances Grouped by SQL Version, Edition – A count of all instances by SQL Version and Edition (ignores Service Pack)

Instances Grouped By SQL Version, Edition and SP – A count of Instances based on SQL Server Version, Edition and Service Pack

Bug Fixes/Minor Enhancements

  • Fixed an incorrectly spelled column.
  • Fixed incorrect calculation of Server number.
  • Added extended properties to all tables.
  • Added header at the top of all Views and Procedures

And All the Rest

This (Simple Database Inventory Manager™) is of course provided free of charge, use-at-your-own-risk. There is no warranty either expressed or implied. If SDIM™ burns down your data center, uninstalls all your favorite toolbars and ruins your best pair of dress socks, I’m not at fault. Remember to back up your databases!

And if you’ve skipped over everything just to get to the link, here it is: SDIMS v2.1

-CJ Julius

Creating a Simple Database Inventory Manager with Powershell – Part IV: GUI Front-End

Last Time, on Inventory Manager…
Now we make things pretty.
Let’s make things pretty.

Our data pull script has run, the database contains all our server  \ instance .database information and flowers are blooming; things are good. If this doesn’t sound like anything you have done, head back to the Introduction to see if you missed something.

Now it’s time to get everything connected so we can just fire up a GUI and press some buttons, to get the data we need fast.

How the Sausage is Made

If you’ve been following along with this series, and you’ve set up everything as instructed, then you should be able to download the pre-made GUI script and run it out of the box. If you’re pointing to a custom instance or database just change the $RepositoryInstance and/or $RepositoryDB before firing it up. If you want to learn more about how this was put together, keep reading. If you don’t care how the whazits work, you’re done.

At the top of our list is to create a form with buttons and give them names so we can call them in our Powershell. You can either build the form manually with this guide here, or use Visual Studio*. I’m going to be using the latter method because it’s the most versatile, and frankly the easiest. If you use the former, then you’re kind of on your own. Sorry.

In the  Visual Studio method (I’ll be using 2013 Ultimate) you’ll be utilizing Windows Forms and then running them through a “cleaner” to make them Powershell ready. This guide at FoxDeploy explains the whole thing spectacularly and shows you how to create some very complicated UI’s that are Powershell-friendly. I’d recommend going through Parts I and II as they’ll be the things you’ll need to create what we’re going to use here and then come back. Don’t worry, I’ll wait.

Then We Build

Got the GUI code? Cool. The first part of Stephen’s code uses a -replace to filter the Windows Form code and make it work in Powershell. I took that piece and made it a second script so I could just have the clean version of the XAML in my final code. You can find that code here.

Just copy/paste your <Window>…</Window> code over the commented area and run the script. It will spit out the final code and tell you all of the objects you can tie actions to (Name, Value). Then drop in Stephen’s XAML reader code to the main script with the cleaned code and you should have a GUI… that does nothing.

Whee.
Whee.

As I mentioned before, when you pushed the XAML code through WPF_to_PSForm.ps1 it will tell you what the objects are on your form. For our purposes, this is simply a few buttons that need to be tied to stored procedures. We do this though .Add_Click() as in the example below:

$WPFBt_All_Data.Add_Click(
{
$sqlCommand = "
EXEC dbo.prGetInventory;
"
$dataset = Invoke-SQL -datasource $RepositoryInstance -database $RepositoryDB -sqlCommand $sqlCommand
Write-Host $dataset
$dataset | Out-GridView -Title "Database Inventory"
}
)

Nothing crazy that we haven’t been doing other than using Out-GridView. This cool little cmdlet pushes datasets out to a customizable table with filtering, sorting the ability to remove columns etc. -Title “SomeTitle” is the window title.

Sample sorted Database List with a few columns removed.
Sample sorted Database List with a few columns removed.

Once you’ve coded all of the buttons, then add the form display at the bottom, using out-null to suppress messages:

$Form.ShowDialog() | out-null

And that’s done. A Winner is You!

What now?

Using these scripts, you can go out and grab any information from the servers\instances you specify, pull it back into a centralized location and then use a GUI front-end to make fine tuning and retrieval easy. As I stated previously, this is a bare-bones system to centralize your database information. You can gather any piece of information from the Server, Instance or Database level by using the same tools that are currently collecting and retrieving this information.

It’s been a long journey, but thanks for sticking with it! If you want to make any alterations to the code or tighten it up (Odin knows that it needs it), feel free to make the changes and shoot them back to me. I’ll definitely give you credit for significant changes in this blog or the code itself.

Also, and I think this goes without saying, but if you want to use this in your personal or business environment: have at it! Just please make sure you give me proper credit, with maybe a link back to my blog/Twitter/Linkedin? That’d be super cool of you.

Thanks again and happy Inventorying!

–CJ Julius

*Full disclosure: I have not tried this with the Community version of Visual Studio, so all the features may not be there.

Creating a Simple Database Inventory Manager with Powershell – Part III: Data Pull

Powershell time; no really.
DB_inven_Mgmt_PS1_sm_III
I come bearing scripts.

Now it’s time to get to get this thing moving. We’re going to go out to each of our server\instances and pull back the information for our tables, updating them with the stored procedures from the last section.

We’re going to be looking at this script [DB-DataPull.ps1]. It’s about as simple as I could get it for our needs. There’s not a lot of frills, but it’s a good cop and it. gets. results.

If you think you missed something you can go back to Part II: Stored Procedures or check out the Introduction.

Get this Jalopy on the Road

The only thing you need to do is specify where the repository is. If the repository is on your local machine in the DBAdmin database then you need to change nothing.

$RepositoryInstance = '(local)'
$RepositoryDB = 'DBAdmin'

After that you’re done. Seriously. The rest of this post is going to be about the nuts and bolts of the script and what does what and why. If you’re looking to just get it fired up then you’re done. Be gone with you.

What’s in the box?

The first few functions (Get-Type and Out-DataTable) are required to turn multi-line WMI-Object output into DataTables so we can insert them into the Repository. These have been cleaned up and/or modified to fit our needs but are based on the code in the two links I provided.

The Invoke-SQL function is a pared-down version of a pretty popular script for sending dynamic SQL directly to a SQL server. There’s not much to be said about this one other than it opens a connection, sends the command and returns the results as a datatable.

Time to get into the meat of the process. First up, let’s grab all the Instance information using the stored procedure we built in the last post.

$ConnectionString = Invoke-SQL -datasource $RepositoryInstance -database $RepositoryDB -sqlCommand "
EXEC dbo.prGetConnectionInformation;
"

Using a foreach loop to cycle through the rows and thus connecting to each instance to pull the information. We’ll remove the ‘\\MSSQLSERVER’ part since that will actually break our connection, even though it’s the name of the instance (For more information on why this is, see every other Microsoft product ever created).

foreach ($Row in $ConnectionString.Rows)
{
Try
{
$SubConnection = $($Row[0]) -replace '\\MSSQLSERVER',''
$InstanceID = $($Row[2])
Write-Debug $InstanceID
Write-Debug $SubConnection
$Version = Invoke-SQL -datasource $SubConnection -database master -sqlCommand "
SELECT SERVERPROPERTY('productversion'), SERVERPROPERTY ('productlevel'), SERVERPROPERTY ('edition'), @@VERSION
"
}
...

And then with another loop we use dbo.UpdateInstanceList to push all that into our database.

Invoke-SQL -datasource $RepositoryInstance -database $RepositoryDB -sqlCommand "
EXEC dbo.prUpdateInstanceList
@MSSQLVersionLong = '$MSSQLVersionLong'
,@MSSQLVersion = '$MSSQLVersion'
,@MSSQLEdition = '$MSSQLEdition'
,@MSSQLServicePack = '$MSSQLServicePack'
,@InstanceId = $InstanceID
"

That’s it for the Instance information, let’s get the database information. We use the same process to generate the connections as we did before, so I’m going to skip that. The only change you should note is the inclusion of the statement TRUNCATE TABLE dbo.DatabaseList since we are going to completely repopulate it. This way no matter if databases are added or removed, we’re starting each pull with a clean slate.

We get our data via a cte…

$DataPull = Invoke-SQL -datasource $SubConnection -database master -sqlCommand "
with fs
as
(
select database_id, type, size * 8.0 / 1024 size
from sys.master_files
)
select
$InstanceID AS 'InstanceId',
name,
(select sum(size) from fs where type = 0 and fs.database_id = db.database_id) AS DataFileSizeMB
from sys.databases db
ORDER BY DataFileSizeMB
"

…and push it into the Repository via our stored procedure.

Invoke-SQL -datasource $RepositoryInstance -database $RepositoryDB -sqlCommand "
EXEC dbo.prInsertDatabaseList
@DatabaseName = '$DatabaseName'
,@InstanceListId = '$InstanceListId'
,@Size = $Size"

Lastly, we’ll get Service and Server information with the same rinse-and-repeat method, with one notable exception. If you try to return the results of a WMI-Object and parse it into a SQL table, then you’re going to have a bad time.

This is where our two functions from the beginning come in to play. Out-DataTable and its sidekick Get-Type return the results into the proper type for our foreach loop.

$ServerInfo = Get-WmiObject win32_Service -Computer $Row[0] |
where {$_.DisplayName -match "SQL Server"} |
select SystemName, DisplayName, Name, State, StartMode, StartName | Out-DataTable

Now, if you run EXEC dbo.prGetInventory on your Repository database, you should see all of the information you could ever want right there. Magic.

But Wait, There’s More!

Now we’ve got all the data in one place, which is nice and all, but what if we want to get this information quickly? Sure we can jump into SSMS and run the procedures that have the data we want. However, I propose we make a GUI front-end so we can win friends and get free drinks.

DB_DataPull_FrontEnd_GUI_2
Something like this?

We’ll do that in Part IV: The Voyage Home GUI Front-End.

–CJ Julius

Creating a Simple Database Inventory Manager with Powershell – Part II: Stored Procedures

Dynamic SQL? Them's fightin' words.
Dynamic SQL? Them’s fightin’ words.

Now that we’ve got the tables built and populated them with data (you did that right?) we can define how our Powershell script is going to access that data. If you don’t know what database or tables I’m talking about, check out the Introduction and Part I.

You can send the dynamic SQL to your database via Powershell but you really want all interactions to go through stored procedures. Why? Lots of reasons, but here’s 2:

  1. Standardization – Every time you call a stored procedure the code will be exactly the same every time. If you change a hard-coded query in one part of your script, but not another, you can get inconsistent results.
  2. Performance – Unless you’re accessing thousands of instances with this script then you may not care too much about performance, but SQL Server has a hard time building query plans off of dynamic SQL; it does so wonderfully when the code is in a compiled stored procedure.

So, that’s all well and good, but what do we need for our Powershell project to do? I’ve created the procedures that will allow you to do the basic tasks that you might be interested in. The sky is the limit if you want to do more.

The Procedure List
prGetConnectionInformation

[prGetConnectionInformation] – Setup script for 2012

Parameters:

  • Not a gosh-darn thing.

Returns:

  • Server\Instance string as ‘Connection’
  • ServerList.Id as ‘Server ID’
  • InstanceList.Id as ‘Instance ID’

Spits out a connection-string friendly version of all the servers and instances in the database.

prGetDatabasesAndSize

[prGetDatabasesAndSize] – Setup script for 2012

Parameters:

  • Not a gosh-darn thing.

Returns:

  • InstanceList.InstanceName
  • DatabaseList.DatabaseName
  • Size of the Database in Gigabytes as ‘SizeinGB’

Returns the database and the instance it belongs to along with the sizes of said DBs in Gigabytes for all databases in the database (Yo dawg, heard you like databases).

prGetInstances

[prGetInstances] – Setup script for 2012

Parameters:

  • Not a gosh-darn thing.

Returns:

  • InstanceList.InstanceName
  • InstanceList.MSSQLVersion
  • InstanceList.MSSQLEdition
  • InstanceList.MSSQLVersionLong

Basically returns everything in the dbo.InstanceList table that someone might be interested in.

prGetInventory

[prGetInventory] – Setup script for 2012

Parameters:

  • Not a gosh-darn thing.

Returns:

  • ServerList.ServerName
  • ServerList.IPAddress
  • ServerList.OSName
  • ServerList.OSServicePack
  • InstanceList.InstanceName
  • InstanceList.MSSQLVersion
  • InstanceList.MSSQLEdition
  • InstanceList.MSSQLVersionLong
  • Size of the Database in Gigabytes as ‘SizeinGB’

Leaves for a while and then comes back with everything in your Inventory. Does not bring back anything from dbo.ServiceList for visual reasons (we’ll see this later in the Powershell).

prGetServerNames

[prGetServerNames] – Setup script for 2012

Parameters:

  • Not a gosh-darn thing.

Returns:

  • ServerList.ServerName

Just brings back all the servers. Nothing else. Stop asking.

prGetServers

[prGetServers] – Setup script for 2012

Parameters:

  • Not a gosh-darn thing.

Returns:

  • ServerList.ServerName
  • ServerList.OSName
  • ServerList.OSServicePack

Returns all the information at the Server Level excluding IP Address.

prGetServerServices

[prGetServerServices] – Setup script for 2012

Parameters:

  • Not a gosh-darn thing.

Returns:

  • ServerList.ServerName
  • ServiceList.ServiceName
  • ServiceList.ServiceDisplayName
  • ServiceList.ServiceStartMode
  • ServiceList.ServiceStartName as ‘Service_Logon’

Spits out everything you and your friends ever wanted to know about Services in the dbo.ServiceList table and the name of the Server they are on.

prInsertDatabaseList

[prInsertDatabaseList] – Setup script for 2012

Parameters:

  • DatabaseName VarChar(MAX) (surprise! parameters!)
  • InstanceListId BigInt
  • Size Float

Returns:

  • Error Code if applicable

Inserts new Database entries as provided by the Powershell script.

prInsertServiceList

[prInsertServiceList] – Setup script for 2012

Parameters:

  • ServerName VarChar(MAX)
  • ServiceDisplayName VarChar(MAX)
  • ServiceName VarChar(MAX)
  • ServiceState VarChar(MAX)
  • ServiceStartMode VarChar(MAX)
  • ServiceStartName VarChar(MAX)

Returns:

  • Error Code if applicable

Wow. Look at all the VarChar(MAX)’s. These could probably be turned into other datatypes that are smaller, especially ServiceStartMode. In the interest of brevity I just made these as versatile as I could.  This may change in the future if I get a chance to tighten this up.

Oh, and this stored procedure inserts new services into the dbo.ServiceList table by finding the ServerName in dbo.ServerList via text search.

prUpdateInstanceList

[prUpdateInstanceList] – Setup script for 2012

Parameters:

  • MSSQLVersionLong VarChar(MAX)
  • MSSQLVersion VarChar(MAX)
  • MSSQLEdition VarChar(MAX)
  • MSSQLServicePack VarChar(20)
  • InstanceId BigInt

Returns:

  • Error Code if applicable

Updates dbo.InstanceList with values passed from the script. If things have changed, this will overwrite the old data in the column.

prUpdateServerList

[prUpdateServerList] – Setup script for 2012

Parameters:

  • IPAddress VarChar(20)
  • OSName VarChar(100)
  • OSServicePack VarChar(100)
  • ServerName VarChar(MAX)

Returns:

  • Error Code if applicable

Adds Server information to dbo.ServerList by matching on the ServerName. This too will overwrite old data with new values passed to it.

Utility.prInsertNewServerAndInstance

[prInsertNewServerAndInstance] – Setup script for 2012

Parameters:

  • ServerName VarChar(MAX)
  • InstanceName VarChar(MAX)

Returns:

  • Error Code if applicable

Utility that Creates New Server and Instance if they don’t exist already.

I Was Told There’d Be PowerShell

And that’s the end of our Database setup. We now have a DB somewhere with tables and procedures, ready to be populated with our Server\Instance.Database information.

With the frame now in place, seats and tires (nice choice of body color maybe), we are prepared to drop the engine in. We’ll get started on that in our next section.

–CJ Julius

Creating A Simple Database Inventory Manager with Powershell – Introduction

Which databases names have the letter “B”?

And then we shall rule the world!
And then we shall rule the world!

All DBAs should keep track of their Servers/ Instances/ etc not only for their own edification, but for Management and security reasons as well. If you’re not, then you need to, as it comes in incredibly handy even if it isn’t a requirement of the job.

Most of the time, this information is compiled into a spreadsheet of some kind or possibly in a word processing document somewhere. Keeping this data up-to-date and accurate is a pain, especially when you have to break it out into multiple tabs and/or over multiple documents.

You could get a full-blown inventory manager that collects and compiles all the data and organizes it for you. But there’s a definite cost to that solution and not one that all companies will find useful (Read: “It’s not in the budget this quarter”).

What if you can’t get someone to shell out the money for a product like that? Then you have to either keep with the spreadsheets (yuck) or you need to find another solution with the tools you have.

What’s the Catch?

A simple frontend for your Database Inventory
A simple front-end for your Database Inventory

So, this is my attempt to resolve this issue using two tools that any MSSQL DBA should have: Powershell and SQL Server. I will point to other software products or versions both paid and free below, but the core code should run using things you should already have. That said, here’s my software list:

Required:

Preferred:

Also I will be making a few assumptions:

  1. Your infrastructure security is set up using Active Directory.
  2. Setting up a new instance is already done and you can connect to it.
  3. Your personal login or the login you are using to execute the code is able to query the relevant system tables and server info on each of the target systems.
  4. You know your current Server\Instance setup.
  5. There is no fifth thing.

What You Get

By the time this series is finished you’ll have a simple GUI front-end (shown above) for current data with all of your servers, instances and all the information you could want about them.

We can pull back and organize any data that SQL Server or Windows Server can spit out including, but not limited to:

  • OS Version, Service Pack.
  • SQL Services running, their statuses and logon information.
  • SQL Server Instance names, versions, and editions.
  • SQL Server Database names and sizes.
  • Ability to dynamically re-pull any of this information if needed.

I will walk you through my solution piece by piece over 4 posts (5 including this intro) that will consist of the following parts. This list will be updated with links to the different sections as they are released.

Part I: Building the Repository Database and Tables
Part II: Creating the Repository Stored Procedures
Part III: Coding the Data-Pulling Powershell
Part IV: Putting together the GUI Front-end

Updates:
2016-11-27
Addendum I: Simple Database Inventory Manager 2.1
2017-05-20
Addendum II: Simple Database Inventory Manager 2.3

See you soon in Part I: Building the Repository Database and Tables

–CJ Julius