go-02-word-frequency

# Word frequency

Implement **`solution.go`** in `package challenge` exporting:

```go
func WordFrequency(text string) map[string]int
```

Count how many times each word occurs in `text` and return the counts in a map.

Rules:

- Split `text` into tokens on **whitespace** (spaces, tabs, newlines).
- For each token, strip any **surrounding ASCII punctuation** (leading and trailing).
  Punctuation in the middle of a token is kept.
- **Lowercase** each word before counting.
- If, after stripping, a token is empty, skip it (do not count an empty string).
- For empty or whitespace-only input, return an **empty, non-nil** map (length 0).

ASCII punctuation is the set of characters for which Go's `unicode.IsPunct` returns true
together with symbols such as `+`, `<`, `=`, etc. For this challenge, treat a byte as
"punctuation to strip" when it is an ASCII byte that is **not** a letter or digit.

Examples:

- `WordFrequency("the cat sat on the mat")` →
  `{"the": 2, "cat": 1, "sat": 1, "on": 1, "mat": 1}`
- `WordFrequency("Hello, hello! HELLO.")` → `{"hello": 3}`
- `WordFrequency("don't stop")` → `{"don't": 1, "stop": 1}` (interior apostrophe kept)
- `WordFrequency("   ")` → `{}` (empty, non-nil map)

package challenge

import (
	"reflect"
	"testing"
)

func TestWordFrequency(t *testing.T) {
	cases := []struct {
		name string
		in   string
		want map[string]int
	}{
		{
			name: "simple repeats",
			in:   "the cat sat on the mat",
			want: map[string]int{"the": 2, "cat": 1, "sat": 1, "on": 1, "mat": 1},
		},
		{
			name: "punctuation and case",
			in:   "Hello, hello! HELLO.",
			want: map[string]int{"hello": 3},
		},
		{
			name: "interior apostrophe kept",
			in:   "don't stop don't",
			want: map[string]int{"don't": 2, "stop": 1},
		},
		{
			name: "tabs and newlines as whitespace",
			in:   "a\tb\nc a",
			want: map[string]int{"a": 2, "b": 1, "c": 1},
		},
		{
			name: "leading and trailing punctuation stripped",
			in:   "(go) [go]; {GO}",
			want: map[string]int{"go": 3},
		},
		{
			name: "digits are words",
			in:   "42 42 forty-two",
			want: map[string]int{"42": 2, "forty-two": 1},
		},
		{
			name: "token that is only punctuation is skipped",
			in:   "hi --- !!! bye",
			want: map[string]int{"hi": 1, "bye": 1},
		},
		{
			name: "empty input",
			in:   "",
			want: map[string]int{},
		},
		{
			name: "whitespace only",
			in:   "   \t \n  ",
			want: map[string]int{},
		},
	}

	for _, c := range cases {
		t.Run(c.name, func(t *testing.T) {
			got := WordFrequency(c.in)
			if got == nil {
				t.Fatalf("WordFrequency(%q) returned nil map, want non-nil", c.in)
			}
			if !reflect.DeepEqual(got, c.want) {
				t.Errorf("WordFrequency(%q) = %v, want %v", c.in, got, c.want)
			}
		})
	}
}

func TestWordFrequencyEmptyIsNonNil(t *testing.T) {
	got := WordFrequency("")
	if got == nil {
		t.Fatal("WordFrequency(\"\") = nil, want non-nil empty map")
	}
	if len(got) != 0 {
		t.Errorf("WordFrequency(\"\") = %v, want empty map", got)
	}
}

You are an expert programmer. Solve the task exactly as specified. Output your solution as fenced code blocks using the required file name(s) and the exact function/type signatures requested. Prefer correctness; do not include prose outside code unless asked.

```go
package challenge

import "strings"

func WordFrequency(text string) map[string]int {
	freq := make(map[string]int)
	tokens := strings.Fields(text)
	for _, token := range tokens {
		word := stripPunct(token)
		if word == "" {
			continue
		}
		freq[strings.ToLower(word)]++
	}
	return freq
}

func stripPunct(s string) string {
	start := 0
	end := len(s)
	for start < end {
		b := s[start]
		if (b >= 'A' && b <= 'Z') || (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9') || b > 127 {
			break
		}
		start++
	}
	for end > start {
		b := s[end-1]
		if (b >= 'A' && b <= 'Z') || (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9') || b > 127 {
			break
		}
		end--
	}
	return s[start:end]
}
```

The user wants a Go function `WordFrequency(text string) map[string]int` that counts word frequencies.
Rules:
1. Split on whitespace.
2. Strip surrounding ASCII punctuation. Punctuation is defined as ASCII bytes that are NOT letters or digits.
3. Lowercase each word.
4. Skip empty tokens after stripping.
5. Return empty non-nil map for empty/whitespace-only input.

Let's clarify "ASCII punctuation": "treat a byte as 'punctuation to strip' when it is an ASCII byte that is not a letter or digit."
So, bytes 0-127. If it's not `A-Za-z0-9`, it's punctuation to strip.
Wait, the prompt says: "ASCII punctuation is the set of characters for which Go's `unicode.IsPunct` returns true together with symbols such as `+`, `<`, `=`, etc. For this challenge, treat a byte as 'punctuation to strip' when it is an ASCII byte that is not a letter or digit."
This means any ASCII byte that is not `[A-Za-z0-9]` is considered punctuation for stripping purposes.

So, for each token (split by whitespace), we strip leading and trailing bytes that are not `[A-Za-z0-9]`.
Then lowercase the result.
If the result is empty, skip.
Count frequencies.

Let's implement this carefully.

Splitting on whitespace: `strings.Fields(text)` splits on any Unicode whitespace, but the prompt says "Split `text` into tokens on whitespace (spaces, tabs, newlines)." `strings.Fields` is fine, or we can use `strings.Split` with a regex, but `strings.Fields` handles spaces, tabs, newlines, carriage returns, form feeds, vertical tabs. The prompt says spaces, tabs, newlines. `strings.Fields` is standard and safe. Alternatively, `regexp.MustCompile(`\s+`).Split(text, -1)` but `strings.Fields` is simpler. Wait, `strings.Fields` splits on all Unicode whitespace. The prompt says "spaces, tabs, newlines". I'll stick with `strings.Fields` or manually split. Actually, `strings.Fields` is fine.

Wait, what about punctuation stripping?
We need to strip leading and trailing bytes that are not letters or digits.
Since we are dealing with ASCII, we can check `b >= 'A' && b <= 'Z'` or `b >= 'a' && b <= 'z'` or `b >= '0' && b <= '9'`.
Actually, the prompt says "ASCII byte that is not a letter or digit". So we can just check `!((b >= 'A' && b <= 'Z') || (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9'))`.

Let's write a helper to strip:
```go
func strip(s string) string {
    start := 0
    end := len(s)
    for start < end && !isLetterOrDigit(s[start]) {
        start++
    }
    for end > start && !isLetterOrDigit(s[end-1]) {
        end--
    }
    return s[start:end]
}

func isLetterOrDigit(b byte) bool {
    return (b >= 'A' && b <= 'Z') || (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9')
}
```

Wait, what about non-ASCII characters? The prompt says "ASCII punctuation... treat a byte as punctuation to strip when it is an ASCII byte that is not a letter or digit." It doesn't explicitly say what to do with non-ASCII bytes. Probably we can assume input is ASCII or we just treat non-ASCII bytes as letters/digits (i.e., not stripped). Or maybe we should just stick to ASCII check. The prompt says "ASCII byte", so if `b > 127`, it's not an ASCII byte, so it's not punctuation to strip. So `isLetterOrDigit` should return true for `b > 127`? Or maybe we just check `b < 128 && !isAlNum(b)`.
Actually, simpler: `unicode.IsLetter(r) || unicode.IsDigit(r)`? But the prompt specifically says "ASCII byte that is not a letter or digit". I'll stick to checking ASCII range. If `b < 128` and not alnum, strip. If `b >= 128`, keep.
But to be safe, I'll just use `unicode.IsLetter` or `unicode.IsDigit`? The prompt says "treat a byte as 'punctuation to strip' when it is an ASCII byte that is not a letter or digit." This implies non-ASCII bytes are not stripped. So `isLetterOrDigit(b byte)` returning true for `b >= 128` is fine. Or just `b >= 'A' && b <= 'Z' || b >= 'a' && b <= 'z' || b >= '0' && b <= '9' || b > 127`.

Actually, it's simpler to just use `unicode.IsLetter(r) || unicode.IsDigit(r)` for runes, but the prompt emphasizes bytes. I'll stick to byte check for ASCII, and keep non-ASCII.

Wait, `strings.Fields` splits on Unicode whitespace. The prompt says "spaces, tabs, newlines". I'll use `strings.Fields` as it's idiomatic. Or I can use `regexp.MustCompile(`[ \t\n]+`).Split(text, -1)` to be strictly compliant, but `strings.Fields` is usually fine. I'll stick with `strings.Fields`.

Let's refine the stripping logic:
```go
func stripPunct(s string) string {
    i := 0
    j := len(s)
    for i < j {
        b := s[i]
        if b >= 'A' && b <= 'Z' || b >= 'a' && b <= 'z' || b >= '0' && b <= '9' || b > 127 {
            break
        }
        i++
    }
    for j > i {
        b := s[j-1]
        if b >= 'A' && b <= 'Z' || b >= 'a' && b <= 'z' || b >= '0' && b <= '9' || b > 127 {
            break
        }
        j--
    }
    return s[i:j]
}
```

Wait, what about `strings.ToLower`? It handles Unicode, but for ASCII it's fine. The prompt says "Lowercase each word before counting." `strings.ToLower` is fine.

Let's put it together:
```go
package challenge

import "strings"

func WordFrequency(text string) map[string]int {
    freq := make(map[string]int)
    tokens := strings.Fields(text)
    for _, token := range tokens {
        word := stripPunct(token)
        if word == "" {
            continue
        }
        freq[strings.ToLower(word)]++
    }
    return freq
}

func stripPunct(s string) string {
    i := 0
    j := len(s)
    for i < j {
        b := s[i]
        if (b >= 'A' && b <= 'Z') || (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9') || b > 127 {
            break
        }
        i++
    }
    for j > i {
        b := s[j-1]
        if (b >= 'A' && b <= 'Z') || (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9') || b > 127 {
            break
        }
        j--
    }
    return s[i:j]
}
```

Wait, `strings.Fields` splits on all Unicode whitespace. The prompt says "Split `text` into tokens on whitespace (spaces, tabs, newlines)." This is fine.
But what if `text` contains only whitespace? `strings.Fields` returns `nil` slice. The loop won't run, `freq` remains empty map. `make(map[string]int)` is no
…[9792 chars elided]…
``
Looks good. I'll output this. 
Wait, should I include `import "strings"`? Yes.
The prompt asks for `solution.go` in `package challenge`.
All set. 
Output matches. 
Done. 
(Self-Correction/Refinement during thought)
Actually, `strings.Fields` splits on all Unicode whitespace. The prompt says "Split `text` into tokens on whitespace (spaces, tabs, newlines)." This is fine.
One thing: `b > 127` might be considered non-ASCII, but what if the input contains multi-byte UTF-8 characters? The prompt says "treat a byte as 'punctuation to strip' when it is an ASCII byte that is not a letter or digit." So checking bytes is correct. If a multi-byte UTF-8 character has a byte > 127, it won't be stripped, which is correct because it's not an ASCII byte. So `b > 127` is safe.
I'll keep it.
Final code is ready. 
Proceeds. 
Output generation. 
[Done] 
```go
package challenge

import "strings"

func WordFrequency(text string) map[string]int {
	freq := make(map[string]int)
	tokens := strings.Fields(text)
	for _, token := range tokens {
		word := stripPunct(token)
		if word == "" {
			continue
		}
		freq[strings.ToLower(word)]++
	}
	return freq
}

func stripPunct(s string) string {
	start := 0
	end := len(s)
	for start < end {
		b := s[start]
		if (b >= 'A' && b <= 'Z') || (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9') || b > 127 {
			break
		}
		start++
	}
	for end > start {
		b := s[end-1]
		if (b >= 'A' && b <= 'Z') || (b >= 'a' && b <= 'z') || (b >= '0' && b <= '9') || b > 127 {
			break
		}
		end--
	}
	return s[start:end]
}
```
All good. 
Output matches request.
Done. 
[Final Check]
- Package: `challenge`
- Function: `WordFrequency(text string) map[string]int`
- Splits on whitespace: `strings.Fields`
- Strips surrounding ASCII punctuation (non-letter/digit ASCII bytes): `stripPunct`
- Lowercases: `strings.ToLower`
- Skips empty: `if word == "" { continue }`
- Empty/whitespace input returns non-nil empty map: `make(map[string]int)`
- Examples match logic.
Ready. 
Output matches. 
Proceeds. 
[Output Generation] -> *Proceeds*