Text first.
Encoding when you need it.

CURRENT DEVELOPMENT IMPLEMENTATION

Count and iterate characters.

Length counts grapheme clusters. Iterating a string yields Char values, including combining sequences and emoji sequences. There is no integer string indexer in this small API.

let text = "Aé👨‍👩‍👧‍👦🇸🇪"
WriteLine(text.Length)
WriteLine(text.GetScalars().Count)
WriteLine(text.GetUtf8ByteCount())

This prints 4 characters, 12 Unicode scalars and 37 UTF-8 bytes.

func ShowCharacters(text: Iterable<char>) {
    for character in text {
        WriteLine(character.ToString())
    }
}

Char.FromString(text) accepts exactly one cluster and faults otherwise. Character literals such as 'é' and '👨‍👩‍👧‍👦' also work. Use GetScalars() for a Sequence<uint> of scalar values and UnicodeScalar for classification.

Segmentation is pinned to Unicode 16. Length scans the text; iteration currently copies a snapshot. Graphemes are not always single displayed glyphs, and concatenation can change boundaries. Equality remains ordinal: precomposed é and e plus a combining accent are distinct even though both count as one character.

CURRENT IMPLEMENTATION

Round-trip valid Unicode.

Utf8.Encode(text) returns Sequence<byte>, with Count, indexed access and iteration. Utf8.Decode(bytes) accepts that same collection interface and returns Result<string, InvalidUtf8Error>.

func RoundTrip(text: string) {
    let bytes: Sequence<byte> = Utf8.Encode(text)
    match Utf8.Decode(bytes) {
        .Ok(let decoded) => {
            if decoded.Equals(text) {
                WriteLine("Round trip")
            }
        }
        .Error(let error) => WriteLine(error.ToString())
    }
}

Use patterns to extract a successful value or its error. For example, RoundTrip("café 🌍") prints Round trip. Malformed UTF-8 produces an error instead of replacement text. Conversion currently copies bytes; it is not a zero-copy API.

RUN THE EXAMPLE

Use the matching development toolchain.

Open the prepared Raven project in VS Code, replace Main.rvn with the complete sample, save it, then run the neoCLR build/run task. Refresh both the reference core and System library when moving from an older workspace. Existing IsEmpty() calls must become IsEmpty.

The downloadable source and expected output are the same files used by the saved-project integration check.

SMALL SCOPE

Where we’re heading.

This is an evolving preview API. We are considering more efficient iteration, text positions for slicing, normalization and language-sensitive comparison. A dedicated scalar value type may replace uint in the explicit scalar view. Other encodings would be conversions at the boundary, leaving the meaning of String and Char unchanged. Possible future Utf8String and AsciiString types could offer encoding-specific operations and guarantees alongside the default text container. Those types and an Encoding hierarchy remain deferred.

Unlike .NET’s UTF-16 Char, neoCLR’s Char can contain multiple Unicode scalars. Numeric character casts and fixed-size character memory assumptions must change. Compiler literal diagnostics currently follow the host Unicode rules; the runtime validates the pinned target rules. CLI constant fields are outside the tested character surface.

.NET offers configurable UTF-8 decoding, including a strict mode. This implementation starts with a strict Result and Sequence-based byte access. Whether those are the right long-term contracts is open for feedback.