It is a mistake to think that you understand anything completely. Something is always missing.
I’m not the first to point this out. In 1233, Zen master Eihei Dōgen (source) wrote:
For example, when you sail out in a boat to the midst of an ocean where no land is in sight, and view the four directions, the ocean looks circular, and does not look any other way. But the ocean is neither round nor square; its features are infinite in variety. It is like a palace. It is like a jewel. It only looks circular as far as you can see at that time. All things are like this.
Right. Everything apprehended by the mind, everything perceived or conceived, is more complicated than it seems. Actual things (objects, events, situations, states, whatever) have vast numbers of details that are not represented by the mind. Moreover, actual things are actually related to other things in vast, perhaps infinite, numbers of ways.
It is like a palace. It is like a jewel.
The ocean is also like water sloshing in a tub, like the hydrocarbon lakes of Titan, like human blood. While the ocean is a great example, vast numbers of things are true about anything, beyond the capacity of any finite set of minds to perceive, or to grasp.
Dōgen goes on:
Though there are many features in the dusty world and the world beyond conditions, you see and understand only what your eye of practice can reach. In order to learn the nature of the myriad things, you must know that although they may look round or square, the other features of oceans and mountains are infinite in variety; whole worlds are there. It is so not only around you, but also directly beneath your feet, or in a drop of water.
Thus, it’s not that there is no reality, or that no true understanding of reality is possible, it’s that understanding is always incomplete. This is so because we are limited in our powers of perception and conception, and because reality is very complicated and interconnected.
Knowledge without Certainty 
Suppose we take this from a computational perspective. How do we choose which features to hold, i.e., how to discretize the knowledge we do have or can have in order to reason about it? Additionally, apart from being able to carve up the world into, say, propositions, how do we decide which propositions are relevant and useful?
> How do we choose which features to hold, i.e., how to discretize the knowledge we do have or can have in order to reason about it?
I don’t think I understand the question. I assume that we have the knowledge we have using representations of some kind, probably discrete representations, perhaps coded with non-discrete intensities. Thus, the knowledge is already encoded with features, and we reason about it using the features it already has. As to how knowledge *should* be encoded so we can reason about it, I don’t know. Maybe: as clearly and concisely as possible.
> how do we decide which propositions are relevant and useful?
There seems to be little cost in learning too much, so we can be exuberant learners. I imagine we focus learning, to the degree that we do, by explicit use in problem solving (learning in order to achieve specific goals), and by experience in the kinds of things that turn out to be useful (e.g., learning people’s names). Also, I suppose there is an instinct, i.e., a drive, in young humans to learn language.
The statement you make… “the knowledge is already encoded with features..” is quite beautiful. “We reason about it using the features it already has” shows the realism that your thought seems to be naturally drawn to. You start at a point which is unconcerned with consciousness, with how these features are represented in the brain etc. and go right to how they are utilized. Your answer to “how knowledge should be encoded” shows the love of clear reality that is, likewise, your natural bent. The great research question here is in defining “concise”. What is the minimum?
A totally acceptable answer, in my opinion, is “the way knowledge is currently represented and has always been represented is very close to it’s minimum concise-ness. The communal mind pulls in knowledge which is presented in the most concise way, but it wants all the features to be there. Thus, Shakespere can be assumed to be knowledge in its most concise form; so can “Good Morning America” and so can Henry James. To compare Henry James to “Good Morning America” we are comparing two different levels of knowledge, depths of knowledge, but the form is already concise.