Sharepoint as a high volume information system

I'm looking at designing some core information systems at a new company I'm working at (described one of my ideas here Workflow system)

I've thought a bit more, and am strongly considering using sharepoint for a lot of the heavy lifting seeing as it comes with so much out of the box.

However, I'm not sure how it will handle the high volume of data we'll be throwing at it. I read the MS whitepaper (http://go.microsoft.com/fwlink/?LinkId=95450&clcid=0x409), and it says about 2000 items in a list is about the limit using traditional design methods.

But first a bit of info on my plan and data structures :

We have multiple clients. Each client has multiple applications. Each application will have multiple, ongoing jobs (or process runs).

Each application will store significant correspondence and documentation. Each job represents the processing of a data file on a single run, and stores information about the job such as the postscript file, postal manifests, etc.

Job volume will be about 50 - 100 a day. Each job will have a workflow, triggered by external programs. Then, say on a "job scheduler" page, production staff can schedule the jobs and perform custom actions on the job (written as plugins).

I was thinking the jobs would sit outside and accessed via the BDC, but I would still like them represented in sharepoint lists, to add in sharepoint functionality and reporting, and they'd be accessible in multiple places

e.g.

Application portal - see jobs for application
Production scheduler - see lists of upcoming jobs, assign to resources, trigger other functionality (e.g. copy print file to printer, produce mailing machine file)
Invoicing view - view completed but uninvoiced jobs, export to accounting package
Client view - client portal displays jobs, invoices, stock levels (from external warehouse system), documentation, change register / helpdesk

So basic info about the job would sit in the BDC, but then sharepoint would capture additional metadata about each job. Also, down the line we might put in more advanced workflows using WF or something like K2 blackpoint / blackpearl.

Is this feasible? Any resources you'd recommend to read to get up to speed?

Solution

To use SharePoint, you should concentrate on what SharePoint is good at and what it is designed for.

SharePoint is a great collaboration portal, it is not so good as a simple high volume database. So...

You can setup a small site for each client and subsites for each job. The goal of the "job site" is to display (using a webpart perhaps) the relevant upcoming jobs, a list of job errors/exceptions and relevant team documentation on each job.

Separate sites can be created to give a particular "view" of the jobs. E.g an "Invoicing" site can be created to give a view again from BDC webparts of what is requiring invoicing.

https://iwsolve.partners.extranet.microsoft.com/SDPS/ may provide some help.

Don't try and store huge amounts of information in a SharePoint list, just because it may be possible to "tag" it with meta data. A database table is perfectly able to include columns supplying additional information if required.

Think about it this way. If you are creating 50-100 jobs a day, putting that data into a list pre-supposes your sites users are going to want to enter metadata on those jobs manually. I thought not, so create systems you need in order to get the metadata stored correctly at source, or store metadata about the "types" of jobs within a SharePoint list and allow SharePoint to match the job type with jobs in the BDC.

SharePoint will help you to integrate all your systems information together, but unfortunately it looks like you have a lot of work to do just planning what information should go where and how each type of use will view it.