Powershell - remove multiple lines of text based on static array?

Question

EDIT: 19/12/13

I failed in appropriately defining the input which may have caused confusion, sorry about that. The input file is a IIS log which the formatting needs to stay intact. The fields look like this; "Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken"

A url GET will show something like this;

2013-12-07 00:23:50 XXX.XXX.XXX.XXX POST / code=5071 80 - XXX.XXX.XXX.XXX Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) http://blah.blah.com/?code=5071 200 0 64 3478

The "code=5071" helps us identify a url, if we strip all of the other ones out we can run a stats utility and find out how many hits, etc where for that one code.

---

Right then, I'm a total newb when it comes to coding so feel free to ridicule. I'm trying to take a log file and strip out lines based on multiple variables, I thought I could create an array and that way there was only one file to edit should a number need to be removed or added. The input file is a simple log containing several fields to which one is a 'ID' so something like; ddmmyy blah blah ID. The ID is a ten digit number to which there are thirty-seven. The purpose is to read the log, strip out all of the non-matching ID's and then output the result to a new log file.

This code works fine but it appears I can only have approximately fourteen "-And" in before it stops

Get-Content .\combined.log | Where-Object{$_-NotMatch '10011250' -And $_-NotMatch '10005816' -And $_-NotMatch '5077'} |Set-Content combined1.log

I trolled the net and learned as much as I could handle on arrays but nothing seems to work and I know it's me and how I set it up. I thought something like this might work;

$a = @(10011250, 10005816, 14200712, 2418, 10005699, 5071, 10001040, 4814, 10025390, 4175, 10005940, 10000040, 10008181)
Get-Content .\combined.log | ForEach($i in $a) {Where-Object{$_-notcontains $a}}| Set-Content combined1.log

As you can tell this is not my area of expertise by a long shot. Any suggestions?

Answer 1

You can test for more than one value in the same -match operation using alternation in your regex. Separate multiple values to match for with the pipe symbol (|).

Get-Content .\combined.log | Where-Object{$_-NotMatch '10011250|10005816'}

will filter out all the lines that match either 10011250 or 10005816.
Also, the -match operator will match against an entire array at once, and return the members that satisfy the condition.

Try this:

$a = @(10011250, 10005816, 14200712, 2418, 10005699, 5071, 10001040, 4814, 10025390, 4175, 10005940, 10000040, 10008181)

$regex = [regex]($a -join '|')

Get-Content .\combined.log -ReadCount 1000 |
 foreach {$_ -notmatch $regex | Add-Content combined1.log}  

for BACON:

$lines = (
'Line containing 10011250',
'Line containing 10005816',
'Line containing 10011250',
'Line containing 10915816'
 )

$lines -notmatch '10011250|10005816'

Line containing 10915816