In the majority of cases, an identity is just a row in a database, typically in a table called something like "users" or "accounts". It could also be a line in a file or some XML in a document or some such too. Even if it a DB, it's not necessarily a SQL DB. It is typically some sort of structured data, though (for example, local machine Windows user identities are stored in the SAM registry hive).
It will have a primary identifier (a username or email address are common, but it could also be a phone number, SSN or other government-issued identifier, physical address, or other sufficiently-unique characteristic[s]). It will most likely have an authentication verifier (a password hash, multifactor token key, etc. are pretty common, but it could also be something more exotic like a smart card public key, TLS client certificate info, or similar). It will typically have some additional information depending on the purpose of the identity (name is common; shipping address and payment info is likely if it's a shopping site; an employer might have stuff like employee number, job title, and manager), often including stuff from the list of potential primary identifiers (almost every online identity wants at least one of phone number or email these days, even if they use something different like a username to actually identify you).
How this database is accessed and used depends on a lot of factors. It could be the back-end DB of a web app, where the DB stores all info relating to any user accounts in tables with foreign keys to the users table but nobody except the web server ever uses the DB. It could be an LDAP server, possibly for an organization's email directory or full-blown Active Directory server, where the DB itself might be in any number of formats but the data is exposed through a standardized interface with any number of clients. It might be the local users list of a PC or mobile device or similar, used directly by the OS and (often indirectly, through OS APIs) by software it runs. It might be internal to a service, but exposed to lots of clients through a proprietary protocol, perhaps in a chat system.
Relatively few identities will have encryption keys - symmetric or asymmetric - associated with them, but it does happen (mostly with systems built for, or around, end-to-end encryption). For example, Signal identities use a phone number as the identifier but a public key as an authentication verifier. An identity in a GPG keyserver obviously contains a GPG public key. It would be quite uncommon for the identity provider to generate a private key, though; typically the key pair would be generated on the client and only the public key shared with the server (in plain text), since that way the server can never know the private key.